As much as we would like to think that our users are like us, they usually are not. To design a good user experience, we need to understand who our users are, what they need, and why. User testing is a type of user research that helps you understand users on a deeper level. But to get the most from this research, it’s important to ask the right questions. A wrong set of questions can nullify the benefits of the interview session and lead product development down the wrong path. Good user testing questions help you gather unbiased user opinions, which ultimately leads to better user experiences. Follow these guidelines on how to conduct user testing with valuable user testing questions.
Define the objective of the user testing
Before you start preparing a list of questions, you need to define the objective of the tests. What do you want to achieve, and why? A clear objective makes it much easier to create a proper list. Generally, user testing can be helpful when:
● You want to validate a hypothesis. You validate a hypothesis by building a prototype and testing it with your users.
● You find issues in existing products. You use existing products and test them with your users.
● You understand how people interact with similar products. As a part of competitor analysis, you ask test participants to interact with products created by your competitors
Each case requires a different set of questions. Use these three types of user testing questions as a foundation for every testing scenario:
- Screening questions
- In-test questions:
- general questions about your product
- specific questions about your product
- Post-test questions
1. Screening questions
Finding test participants is the first step in any user testing. You need to create a list of criteria that defines who is an ideal participant and evaluate all your candidates according to this standard. The user testing questions that you ask during the screening process should help you understand both the demographics and experience of your candidates.
Screening is the process of selecting the right test participants.Here are a few questions you should consider asking during the screening process:
How old are you?
Confirming someone’s age is important because participants in the wrong age range could make test results unreliable. This question will help you verify that test participants are within the desired range.
What’s your highest level of education?
The background of your test participants will inform their actions and opinions as they interact with your product. Education provides insight into the type of work a test participant may be doing and how they do it. For example, people with a higher technical education likely use critical thinking in their work, and this skill makes it easier to interact with the product’s interface. In many cases, they will be able to understand how a feature of a product works, even when this feature isn’t designed very well or doesn’t have a proper introduction.
What’s your income level?
If you offer paid services on your website, you need to ensure that test participants can afford the service. It doesn’t make much sense to invite test participants to interact with a product that offers paid services when they won’t try this product in the future (just because they won’t be able to afford it). The test results will be highly biased.
What habits do you have?
Questions like “Can you tell me about your hobbies?” or “What does your typical day look like?” let test participants talk freely about their likes or dislikes. These questions are icebreakers, and at the same time, they help you gain more insight into your participants’ behaviors and reveal if they have any activities relevant to your research. Ask any lifestyle questions that are related to your topic or product.
How much time do you spend online?
Depending on your objectives, you might need users with different levels of experience with digital products. Generally, the more time people spend using digital products, the higher the level of their tech-savviness (familiarity with common interaction techniques). Experienced users are usually focused more on evaluating features and providing deeper feedback.
What do you use the website or app for?
This user testing question will help you understand which services and related features the users interact with the most on existing apps or sites. Suppose you want to conduct user testing of your e-commerce app. What you might want to know is how confident the test participant is with online shopping tasks. If this person isn’t interested in online shopping, you may want to find another candidate.
Which device do you typically use?
The answer will determine which devices (desktop, tablet, mobile) users are most comfortable using. This information will be helpful when you test a particular type of interaction, for example, how well your eCommerce website works for mobile users. If a test participant prefers to use desktop for web browsing and, in very rare cases, switches to mobile, test results with a mobile version of your design won’t be representative for this user.
How often do you [do something]?
This question will help you evaluate the user’s experience level with digital products. For example, if you test an e-commerce app, you might want to ask, “How often do you shop online?” The answer will help you identify the person’s familiarity with online shopping as well as their buying habits.
2. In-test questions: general questions about your product
These user testing questions will help you to uncover how much users know about your website or app. You can ask the following questions either via an online questionnaire or through a moderated interview.
What brings you to our product?
This question will help you find what drives people to your product. You will also understand what goal the user wants to achieve by using your product. It is an excellent question for uncovering if they have any previous experience using your website or app.
What was your overall impression of [x]? Rate it on a scale from 1 (Dissatisfied) to 5 (Satisfied).
This question will solicit feedback on the user’s general opinion of your product. Once participants answer and provide their responses, you can follow up with “why” to allow them to elaborate on their thoughts.
Have you used a similar product before?
This question will help you identify well-known competitors. It can be insightful to hear who test participants consider your competitors, which may be different from the ones you’ve already identified as such or didn’t think of as competitors.
What prevents you from completing a task?
When you see that your test participants cannot complete certain actions or operations, ask them about the problems they face along the way. This query will help you identify the barriers that users face while interacting with your product. The barriers might be something as simple as a lack of important information or more complex stoppers such as dead-end pages (pages that prevent users from completing a task because the next step is missing).
3. In-test questions: specific questions about your product
These user testing questions will help you discover important insights about user behavior when they interact with your product.
What features do you find valuable and why?
Not all of the features that you offer are equally valuable to your target audience. Sometimes it can be surprising to know that a relatively insignificant feature offers a lot of value to users. This question will give you tremendous insights into what the majority of users find the most valuable about your product.
What parts of the product do you use the least and why?
This question will tell you what parts of the product users can live without. It will help you prioritize development efforts and focus on the area of the product that provides more value.
Is this website/app easy to navigate?
Navigation design is a top priority for any digital product. It’s vital to determine what issues users encounter when they navigate on your website or app. Based on these insights, you’ll be able to introduce changes in your information architecture to make navigation easier for your users.
Can you give me a few examples of real situations when you would do [something]?
When test participants give you specific use cases, the answers become less generic and more accurate. Sometimes people you interview will offer scenarios that you’ve never even explored.
What do you think about this design?
Ask this question when you want to learn more about how users feel about your website or app. Open-ended questions like this one are great for collecting personal opinions about specific aspects of your design. It’s possible to ask this question when you want to get a general opinion about your page/screen or individual elements such as icons or microcopy. It’s easy to turn this qualitative feedback into recommendations for your design.
What convinced you to take an action?
You might want to ask this question if the user decides to complete a certain operation on your website/in your app (i.e., sign up to email, purchase a product, etc.). This question will help you identify hooks that persuade users to complete certain operations.
I noticed you did [an action]. Why?
This contextually relevant question comes from observation during the test. Ask it when you want to have a better idea of the thought process behind the user’s action on the site.
4. Post-test questions
The moment right after the user testing session has been completed is a perfect time to ask test participants if they have questions or anything to add so you can collect more of their personal opinions.
If you could change one thing in this product, what would it be and why?
This compelling question allows you to identify the top-priority change for your test participants. It might be a feature they miss or more intuitive navigation. The insights you get from this question will help you make your product better for your target audience.
What do you expect to see in our product in the future?
Test participants might want to share their thoughts about a feature that they want to see in the future. This feedback will help you prioritize your backlog.
What to remember when asking user testing questions
People interpret user testing questions differently based on how they are worded. Leading questions (questions that lead participants to specific answers) can be a huge problem for user testing sessions because they can cause incorrect insights. Here are a few examples of leading questions:
● Do you like our new design more than the old one? This leading question implies the new design is better than the older one.
● Does the app help you work more efficiently? Test participants will focus on finding the relevant example of work efficiency.
● When do you think that feature X would be helpful? This question focuses users on thinking about specific cases for the feature.
Remember that questions should guide the conversation, not lead it. The goal of user testing questions is to have discussions with the participants, not just complete a formal question-and-answer format. That’s why it’s also important to ask follow-up questions such as “Why do you think so?” Even when users have specific thoughts or opinions, not all of them are conscious of the “why” behind their thoughts/opinions. A casual conversation where the moderator is genuinely interested in hearing a personal opinion will put participants at ease and result in more valuable insights.
Use the “Five Whys” technique, which involves asking “why” five times to understand the interviewee’s intention or point of view.Resist the urge to judge or educate your interviewees. Your interviewees are here to teach you something, not the other way around. It’s counterproductive to judge test participants or try to educate them during the testing. Your goal is to elicit as much information as possible during the time that you have; you’ll have plenty of time to reflect on their perspectives later.
Try to record the answers to user testing questions on video. Video recording can be an excellent way to collect information because it allows you to focus on the interviewee and visually capture every detail of the session. Of course, any time you want to record your participants, you need to ensure that they are okay with that. Always get written permission before starting the recording and be ready to abandon it if your interviewee feels uncomfortable.
Recording the interview session will allow you to refer back to it later.Conclusion
Creating a good UX design test is the result of good user research. User testing questions play a key role in learning about the user experience. Users should be involved in the design process right from day one, and UX practitioners should conduct user testing regularly. Using a Build-Measure-Learn cycle is the only way to create a user-focused product. The questions you ask during the “measure” part will affect all other steps of your design process.