Yesterday, I took part in an amazing workshop.
The organizer brought in dozens of people from the market we’re interested in, and we spent hours talking to them—learning how they think, what they know, and so on.
I’m not going to give away the process, but I do want to touch on some things that kept going through my mind during the day:
Beware of making decisions based one or two data points.
And…
“Listen not to what I say, pay attention to what I do.”
Let me dig into each of these a little.
You see, each of us talked to about 15 or 20 potential customers throughout the day. That’s a good number, but what are the chances that these people are perfectly representative of the people we’re building a product for? Not very high.
Which is why all of the new insights that you have should be validated by looking at market data and cold-hard-numbers. What are most people actually doing? This is common sense, but you would be surprised by how often I see it disregarded.
Treat conversations with people from your market as a way of crafting new questions and figuring out which data you need to look at… but don’t let your interviews be ends in and of themselves.
For example, let’s say that when asking someone about their shopping habits you learn about a new type of vending machine. Don’t just assume that this new type of vending machine is successful or important, but write it down on your notepad and look into it later. Check out the sales figures. See what the clientele looks like and whether they overlap with your target group.
This actually hints at what user interviews can be really good for: exploration. You probably already have great data on what people are doing, buying, and what’s “normal” in your market… so each new interview/conversation you have is an opportunity to talk to people and see whether or not they have any weird habits or quirks that you can learn from.
This is why I think one of the best ways to do user interviewing is to actually seek out strange individuals. People that can expand your vision of what’s possible.
Building a video-streaming product?
Go out and find the most obsessive, crazy binge-watchers. Talk with them. Learn how they pick movies. Have them show you their queues. See if there’s anything that they do that you think would be beneficial to a normal, less extreme audience.
Anyways, the point is that you shouldn’t make wide-ranging strategic decisions based on what you heard from a couple of people in user interviews (always validate with #s)… but you should use them as inspiration for further exploration.
In short: don’t generalize from the 2 or 3 people that said something striking.
Now, let’s dig into the second thing I said earlier:
“Listen not to what I say, pay attention to what I do.”
This just means that you should take what people say with a huge grain of salt… especially “why” explanations.
“Why did you like it?”
“Why did you pick it?”
etc.
As I’ve written before, we are really really bad at understanding why we do things, but will always put forward an explanation when asked.
“Why did you choose Cheerios?”
“Because I love that it doesn’t have much sugar, provides me with fiber, blah blah…”
Or could it be the fact that it was right at eye level in the supermarket (easy to see), and that you were just in the mood for something with that texture during your shopping trip?
Who knows…
What we do know is what they did. They purchased Cheerios.
Their purchase data (knowledge of their actual behavior) is worth 1000x more than anything they say about “what they want”, “what they’d buy”, or “what they like”. Their preferences are revealed through what they fork out cold, hard cash for.
That’s where I think the economists got it right.
So the best way to really gain insights into your market from conversations would be to talk with people about what they did.
“What are the last 5 things you purchased? What else do you buy in the breakfast aisle? Jam? Interesting, how often do you purchase that?” And so on…
Most of what you hear will align with what you already know, but every once in awhile you’ll hear something a bit surprising—a bit weird… and that’s where the insights are. That’s where big breakthroughs come from.
So, to recap: Don’t assume the people you’re talking to are representative of your target market. Use them to get new ideas and help you explore your product-space–but always validate what you hear with hard data.
Try seeking out extreme people that you can learn things from. Expand your vision of what people can do in the area you’re working in.
Finally, pay attention to what people do. Don’t ask them why questions (for explanations). Just get the info on what they do and how often they do it.
Everything else is suspect.
Until tomorrow,
Jason
