Introduction
Behavioral science firms have exploded in popularity over the past 15 years, largely because of the popularity of books like Richard Thaler and Cass Sunstein’s Nudge, which covered research from the behavioral sciences on how an understanding of human decision making could be used to drive large business and public policy results.
However, now that we’re well over a decade into the applied behavioral science era, we have enough real-world data to determine whether or not nudging and the applied behavioral science firms that use them are effective, and the results aren’t pretty.
The most comprehensive study to date on the impact of nudges in real-world contexts shows that they have, on average, a 1.4% impact on the outcome variable. This is 1/6th of what the relevant academic studies predict (8.7%). A more recent re-analysis of nudging effect sizes published in The Proceedings of the National Academy of Sciences found a similar result. As they summarized: “…our Bayesian analysis indicates that, after correcting for this bias, no evidence remains that nudges are effective as tools for behaviour change.”
Given all this, it’s quite apparent that applied behavioral science as popularly practiced is much weaker than believed.
While many behavioral science firms publish seemingly impressive nudging case studies touting 40%, and sometimes 100+% impacts, these are often based on cherry-picking and small sample sizes. The fact of the matter is that the bodies of research most of these firms base their interventions on have been shown to either be false or much less powerful than originally reported.
Does that mean the behavioral sciences are useless in a business or public policy context?
No. But it does mean that they are being used inappropriately.
What works and what doesn’t
I’ve been a practicing applied behavioral scientist for close to 14 years, and I’ve worked on behavior change projects for dozens of companies and organizations, including the World Economic Forum. During that time, I’ve tested most of the promising behavioral science research, including that from behavioral economics. While I have an academic background, I’ve always been a skeptical pragmatist, focusing on results. If research doesn’t help us built better products, marketing campaigns, or companies, I’m not interested in it.
Early on in my career, I realized that nudge-style interventions didn’t work. At the time, I was consulting a variety of different companies, large and small, and was focused on figuring out “behavioral bottlenecks”. These were limiting factors that were preventing behaviors from occurring. For example, motivation or knowledge might be behavioral bottlenecks for exercise behavior. If we can determine the bottlenecks preventing someone from doing a behavior, we should in theory be able to remove them and cause a behavior change.
Most of my analyses for clients showed that there were big issues preventing their customers from using their products or following through on desired behaviors. For example, one of my clients was building a self-improved focused social network. After doing a behavioral analysis of their product, and talking with users, it became quite obvious to me that the product wasn’t working because the activities we were asking people to do were too hard or not motivating enough. However, making changes in these two areas would have required major surgery, and so we ended up implementing smaller, nudge-style interventions and usability enhancements. A lot of our work focused on creating better notifications, to cue people to re-engage with the app. Needless to say, these only had a marginal impact on engagement and retention. The company ended up fizzling out and closing down.
After working with a number of other companies, and implementing a variety of different nudge interventions, it became clear to me that this view of behavior change (small tweak-> large result) was unrealistic. In fact, as early as late 2011 and early 2012, I had a strong suspicion that a decent portion of the research out there was either poorly done or fraudulent. These suspicions were vindicated in the following years by a variety of new studies and developments. Most notably:
-
2011-2015: The Center for Open Science’s Reproducibility Project showed that only 36% of psychological science studies could be replicated.
-
2012: Behavioral priming research, heavily covered in Kahneman’s magnum opus, Thinking Fast and Slow, had major replication failure.
Since 2015, the bad news about the behavioral sciences has continued to pile up:
-
2018: Dan Ariely’s most cited academic paper, and one of the key pieces of behavioral science research used by insurance giant, Lemonade, failed to replicate.
-
2018: Camerer et al found that 62% of studies could be replicated, but that the effect sizes were 50% lower than originally reported.
-
2022: An even more recent study looking at pre-registered research found that only 50% of studies could be reproduced.
This is all to say that, as of 2022, it is quite clear that there is a major problem with the behavioral sciences. This isn’t to say that there isn’t any good research out there, but a lot of the studies that have been used to justify nudge-style interventions have either been overturned or found wanting.
The fact of the matter is: Small changes generally result in small results.
If you’re looking to hire a behavioral science firm to make small changes and large results, save your money.
If you’re willing to make substantive changes in search of large results, then you should instead work with a Behavioral Strategist—who can help you come up with a behavior change strategy based on your business goals and an in-depth understanding of your target market/user group.
This is the only approach that results in robust, company-changing outcomes.
What is Behavioral Strategy?
Behavioral Strategy is an interdisciplinary approach to aligning business objectives with target market behavior. In other words: it’s the process of choosing a behavior that is doable, useful, compelling, and enjoyable for the target market that also leads to an important business outcome for the company.
Once we’ve done our research and chosen the right behavior for our target market, we can be confident that any product or service we create around that behavior is more likely to succeed.
A quick example: Let’s imagine that you work for a digital health company and are planning on creating a “health assistant” app to help people lose weight and become healthier. Your target market is individuals above 60 with a chronic health condition.
How are you going to determine what features to build? How are you going to construct the user experience?
If you’re like most product or strategy people, you’ll first do a review of the competitive landscape. You’ll download all the available products, look at the features, read reviews, talk to experts, and talk to users. After doing this, you’ll have a sense of which features are available in the market, which features seem to be liked, which features seem to be loved, and which features leave something to be desired. You’ll also have a laundry list of all the things people in the market seem to desire. You use this information and your creativity to come up with a concept, and work with a team to build out a minimum viable prototype you can test.
Sounds reasonable, right?
It is. But it’s also likely going to result in a suboptimal product with low user retention and minimal impact on company goals or KPIs.
Here’s why: this approach focuses the team on the wrong thing—features.
Features are merely a means to an end. They are ways users can perform behaviors. A good product allows users to do certain behaviors more enjoyably and easily, but it is merely a channel by which users can perform certain actions and achieve the associated results in an expedited or enhanced way.
Focusing on features is focusing on the wrong thing. The right thing to focus on is behavior.
Focusing on behavior
If we want to achieve business goal X, what user behaviors can get us there?
Out of all these potential behaviors, which solve a true problem for our target user group? Which behaviors can they do easily? Which behaviors are they excited by? Which behaviors do they enjoy?
Matching our goals up with the user behaviors that also align with their problems, interests, likes, and abilities results in a win-win. If we create a product or service around an aligned behavior, we’ll achieve our company goals and they’ll solve a problem in an enjoyable manner.
Our product or service can take dozens of different forms. For example, a weight-lifting product or service can take all the following forms:
-
Weight-lifting coach (via video call)
-
Weight-lifting accountability coach (via text or video call)
-
Smart weight lifting assistant (AI/app based)
-
Smart weight lifting assistant (wearable)
-
Weight lifting subscription (supplies/weights delivered to your house as you progress)
-
Weight lifting gym
-
Gym-in-a-box (full home gym delivered to you)
-
Weekly weight-lifting instructional newsletter
-
Weight lifting Youtube channel (with tutorial videos)
-
Weight lifting social network (gym/accountability buddies)
-
Weight lifting progress tracking app (take photos over time, etc.)
-
Home gym finder (Airbnb for home gyms)
-
Group weight lifting app (find local workout groups and sign up for workouts)
-
Etc.
This is just a small sample of the possible products or services that can be created to encourage weight lifting. The specific one we would choose would be based on further user research and our company’s strengths and capabilities. But as you can see, there are many different ways that a company could enable a desired behavior.
If our company creates a good user experience that enables the high-value and compelling behavior we’ve chosen, we can be quite confident that we’ll be successful in creating something engaging that accomplishes our stated business goal.
Saving your money
So, what’s the answer? Should you hire a behavioral science firm?
Probably not.
You should, instead, be working with a Behavioral Strategist who can help you focus on behavior and align it with your company goals in order to create a successful product or service.
A Behavioral Strategist will work with you to understand your business goals, and then help you match those up with the user behaviors that will accomplish those goals in a way that’s enjoyable for your target users. By focusing on behavior instead of features, you’ll be much more likely to create a product or service that engages your customers and is successful in achieving your business goals.
