Or why you should at least take them with a HUGE grain of salt…
“The truth is, not only are people pretty poor at recognizing what will influence their future behavior, it turns out that they are also not that well attuned to what persuaded them after the even either. As part of a TV news magazine program one of us was asked to assist with a segment of the show that sought to identify the reasons why people might be persuaded to help others in a series of everyday (nonemergency) settings. At a busy New York City subway station we hired researchers to count the number of commuters who donated to a street musician as they walked past.
After a short time a small change was made to the situation that had an immediate and impressive impact. Just before an approaching (and unsuspecting) commuter reached the musician, another person (who was in on the act) would drop a few coins into the musicians hat in view of the approaching commuter. The result? An eight-fold increase in the number of commuters who chose to make a donation.
In a series of post-study interviews with commuters who did donate, every one of them failed to attribute their action to the fact that they had just seen someone else give money first. Instead they provided alternative justifications: “I liked the song he was playing”; “I’m a generous person”; and “I felt sorry for the guy.”
That people are generally poor at recognizing the factors that influence their behaviors both before and after an event raises an immediate implication for any business or organization that invests time, effort, and often considerable dollars asking their customers and clients what actually drives their buying decisions and behaviors. Although we are confident that many customers will happily provide answers, we are less confident that the answers they provide will be an accurate reflection of what happens in reality, resulting in marketing strategies based on those answers having high failure rates.”
— Steve Martin, Noah Goldstein, and Robert Cialdini
(The Small Big, pages 4–5)
Is this to say that you should never interview or survey your users?
I don’t think so. But I do think that you interview and survey questions should be geared towards figuring out what their specific behaviors in a given domain are, instead of figuring out *why* they do certain behaviors.
See yourself as a private detective, a fly on the wall. Your job is to take notes on what the people you’re interested in *do*, not what they say they do. Be an observer, not a psychoanalyst.
For example, you should ask them questions like:
- Can you tell me about the last thing you purchased online? What was it? What was your process for finding it, and then purchasing it?
- Are you comfortable with showing me your order history on Amazon?
- etc.
Through these types of questions you can learn a lot of useful information about their current behavior patterns. Since most people don’t deviate very much from their current behavior patterns, this information will be quite useful to you when you’re planning out your product — since it needs to fit into the lives and habits of your potential users.
You should not ask questions like:
- What do you like about online shopping?
- Why did you click on what you just clicked on?
- Why did you buy the last thing you purchased?
- If we gave you access to this product, would you use it?
The answers you get from these questions are likely to be almost complete BS. People are amazing at coming up with just-so stories to explain their behaviors, and will gladly conjure up a tale to satisfy you, the interviewer (even if they don’t realize they’re doing so).
If you want to learn more about the proper way to do user testing (and interviewing), check out this article. It will provide you with a step-by-step guide to do quite useful user observations and interviews.
