Cognitive biases can have a tremendous impact on the product design process. When UX practitioners aren’t aware of their own biases, they can end up falling into the trap of faulty conclusions. A systematic error in thinking caused by cognitive biases can affect the judgments and product design decisions that team members make.
Cognitive bias can happen when designers build a hypothesis on how something should work for their audience and want to validate this hypothesis through user research. Overcoming cognitive biases during user research will help you to improve the efficiency of your product design. Below are some common cognitive biases and practical tips on how to deal with them.
What are cognitive biases?
When processing information, our brain naturally creates mental shortcuts. These shortcuts reduce cognitive load for our brain, so every time it must process new information, it simply uses existing shortcuts to do it faster. It’s easy to think of cognitive biases as something that help people make sense of the world. However, this “feature” can cause a lot of problems because, under certain circumstances, mental shortcuts can easily lead us to incorrect assumptions.
Eight types of cognitive biases
There are dozens of cognitive biases that take many different forms. I will take a closer look at eight of the most common types of cognitive biases that pop up when interpreting feedback from user research.
1. Confirmation bias
People tend to give more weight to evidence that confirms their assumptions and to discount data and opinions that don’t support those assumptions. Psychologist Daniel Kahneman, who introduced the concept of confirmation bias, says that confirmation bias happens “when you have an interpretation, and you adopt it, and then, top-down, you force everything to fit that interpretation.”
Confirmation bias is perhaps the most dangerous bias from this list because it severely affects your approach to UX design. It prevents UX practitioners from being open-minded and can cause a lot of problems during ideation and brainstorming sessions. Practitioners who suffer from confirmation bias tend to strengthen their beliefs in the face of contrary evidence. During user research, you might ignore pain points that users are suffering from because those points simply don’t fit with your existing assumptions. For example, a UX practitioner might hear users complain about a poorly designed navigation system in a product. A UX practitioner might discount such feedback because the design looks logical to them.
2. False-consensus bias
False consensus is the assumption that other people will think the same way as you. It is a very dangerous bias in the early stages of product development because a team might invest time and effort in building something that doesn’t have much value to users, all because they are confident in their idea. It’s possible to minimize the risk of false-consensus bias by working more with your target audience, their needs, and their wants. You need to identify and articulate your assumptions and be ready to validate them with your real or potential users.
3. The recency effect
People tend to give more weight to their most recent experiences. UX practitioners who suffer from the recency effect tend to form new opinions biased toward the latest news. For example, when a UX researcher conducts a series of usability testing, they might focus more on the problems found in the latest session.
4. Anchoring bias
When people make decisions, they tend to rely on information that they have at the moment. For example, when a product team releases the first version of a product and wants to improve it, they usually evaluate the second version by comparing it to the first one. This bias can create a trap for user researchers because they might be too focused on the information they received before doing user research. It’s worth remembering that things you learn before, during, and after user research should have equal weight.
5. The peak-end rule
People tend to judge an experience more on how they felt at its most intense point (snapshot one) and its end (snapshot two) rather than based on the total sum or average of every moment of the experience. The most intense point can be the most memorable experience that you have with a product. For example, the first call on your new phone to your best friend will elicit a memorable experience whether it’s pleasant or unpleasant. The remembered value of snapshots dominates the actual value of an experience.
The peak-end rule explains how we remember the experience. The peak in this example is positive, but it can also be negative.6. Social desirability
People tend to make more “socially acceptable” decisions when they are around other people. A person’s behavior might be completely different when they are left alone and acting independently. As a researcher, you need to be aware that the answers you get during interviews might not be valid because test participants want to give desirable answers. Users can experience social desirability bias even if it’s not how they really feel. As much as possible try to observe users in their real environments with the same conditions in which they would be using the product.
7. Clustering illusion
Data clustering is a process of organizing large amounts of data into groups or themes based on their relationships. Many UX research books suggest that, in order to make data-informed design decisions, UX practitioners should always cluster qualitative and quantitative data sources. Data clustering works well for experienced researchers, but it can create a trap for people who recently started in the field of UX research. Many beginners make false clusters when they analyze data and tend to see patterns even when there aren’t any. In many cases, the root of this problem is because the sample size is too small. A small sample size makes it harder to understand whether the user behavior is typical for larger user segments, increasing the risk of an incorrect assumption. Only when you have a sufficient sample size can you cluster data and suggest data-informed changes.
8. Framing effect
People react differently to the same information depending on how it’s worded. The framing effect is especially noticeable when we ask questions: “Do you enjoy this feature?” is an example of a leading question that frames the answer around the word “enjoy.” By asking this question, a person is likely to think about only the positive parts of the product experience. Avoid asking leading questions at all costs. The question “What do you think about this [product/feature/design/]?” works much better because it allows the user to think outside one particular part of the experience (either positive or negative) and focus on their genuine opinion of the product instead.
The Nielsen Norman Group (NNGroup) shares an interesting example of how the framing effect influences UX practitioners. According to the NNGroup, UX practitioners obtained usability testing findings and were asked a simple question: “Should a search function be redesigned based on usability testing findings?” The task results were communicated in two different ways, with half the group receiving the information in the first way, and the other half receiving the information in the second way:
- 4 out of 20 users could not find the search function on the website.
- 16 out of 20 users found the search function on the website.
UX practitioners who received this feedback reacted differently depending on which way the information was phrased. 51% of respondents who received the negative statement (the first statement above) wanted to call for a redesign. At the same time, 39% of respondents who received the positive statement (the second statement above) did not feel a redesign was necessary. This proves that people can interpret the same feedback differently (in a positive or negative light) depending on how this feedback is worded.
Result from research conducted by Nielsen Norman Group. Image by NN Group.Tips for overcoming cognitive biases when doing user research
Now that you know common types of cognitive biases, it’s time to share practical tips on how to overcome these biases before beginning user testing.
Be explicit about your assumptions upfront
It’s natural for any person to have assumptions, and UX researchers are no exception. But when you are doing user research, you should always keep in mind the assumptions you have (both general assumptions and specific assumptions related to a particular project). You can avoid many types of cognitive biases by becoming more self-aware of how you look at data. Thus, before you start to research, invest time in identifying your assumptions—list them out, and share this information with your team to help overcome them.
Make sure your sample size is large enough
Small sample sizes are one of the things that can make your research unreliable. But how many test participants do you need for a usability study? Is it enough to test your product with five users? In reality, it’s impossible to give a one-size-fits-all answer to this question. A rule of thumb is to do your best to get a large sample size filled with users who represent all groups of users in your target market. By doing that, you won’t lock yourself into one group’s preconceptions. For more practical tips on this topic, I recommend reading a Nielsen Norman Group article on selecting a sample size for usability testing.
Define the process of analyzing results
What criteria will you use for defining actionable insights? What metrics do you plan to use to analyze user behavior (i.e., ideal time-on-task, expected bounce rate, etc.), and how will you categorize user behavior quantitatively (i.e., what percentage of test participants should successfully complete the flow)? You should specify clear criteria before starting user research because well-defined criteria is much easier to analyze and categorize data. For every piece of feedback, ask yourself how you’re framing the data. The awareness of a process will help you understand why your perspective feels positive or negative.
Check your mood
People tend to underestimate how much their emotions affect their attitudes and behaviors. Your emotions have a direct impact on your research findings (this phenomenon is known as an empathy gap, and it’s also a type of cognitive bias). If you feel sad, you may have trouble relating to someone who feels happy and excited. And while it’s hard to control your emotional state, you can still at least write what you feel before starting the session. This information will help you analyze the research results and separate your emotional state from the qualitative data collected during the interview.
Don’t talk too much
Let your test participant speak instead. Listen to them and observe their reactions. Ask clarifying questions such as “Why do you think so?” to let interviewees express their ideas. Whenever you have a pause, don’t try to fill the silence, let the participant do it for you.
Watch your body language
Good user researchers are neutral in their reactions (they never show their emotions). And they do it for a reason—both positive and negative responses from a researcher can have an impact on user research findings. Test participants won’t suffer much from social desirability bias if you hide your emotional reactions during the sessions.
Conclusion
Overcoming cognitive biases is an essential part of UX practice. Interestingly, UX practitioners who believe that “these biases affect other people, not me” suffer from blind-spot bias. Blind-spot bias is the failure to notice your own cognitive biases. The danger of this bias is that it may cause you to ignore style issues when designing products. That’s why the first rule of product design is simple—be open-minded. It’s essential to be aware of your own biases and be ready to fight them.