Illustration by Tracy Dai
Design is a universal language that can bind human emotions to an electronic device. These devices and their respective applications solve problems that humans encounter, so it’s important to design usable products to help people find meaningful solutions.
Understanding how usable a design is can help designers measure engagement; however, many organizations spend little to no investment in measuring the usability of the products and services they provide, as usability testing can be expensive.
If your organization has enough capital to invest in qualitative user research methods, you can gain better insights into how usability impacts the product. Let’s explore how this testing works and how usability metrics can give designers an upper hand in creating a meaningful and usable application.
The usability framework
The usability framework is a tool centered around a user who will interact with a particular product or a service in a specific context. It measures the usability of a product or service by capturing and creating benchmarks of success for effectiveness, efficiency, and satisfaction:
- Effectiveness is the accuracy and thoroughness of the goal achieved.
- Efficiency refers to the resources the user exhausts to achieve a particular goal.
- Satisfaction relates to the user’s subjective thoughts on their experience using the product. This includes their opinions and attitude on level of comfort, relevance of the application, and applicability of use.
If a product does not achieve the goals of effectiveness, efficiency, and satisfaction, it will not achieve the usability goals of the product—or the user’s goals—either.
Why should we measure usability?
Usability underpins everything on an application. If the application has poor usability, users will not be able to interact with it properly. Alternatively, users will come back to an application again and again if they can achieve their tasks quickly and easily, understand the functionality without experimenting, and discover the features of the application with ease. Measuring usability helps designers pinpoint what isn’t working (e.g., an application can be un-engaging, poorly written, or unattractive) and provides an opportunity to improve the work.
The ideal usability of an application differs from one focus group to another. When designing applications, we focus on the set of people who will eventually use it. These people have their own behaviors, demographics, pain points, and goals. To identify these target groups, a designer can create personas to help focus on which usability metrics can be used to create a delightful experience.
Through usability metrics and testing with these personas in mind, a design can be verified as usable or not. They can also:
- Track usability improvements. Designers can see how the usability of the application is improving between iterations of the product release.
- Improve communication between a focus group and the design team. The usability metrics will help designers understand the problem areas that they need to address and how they can best approach the focus groups.
- Better understand product positioning. Having a competitive advantage over similar products is necessary. Usability testing helps a product team focus on the unique value proposition of the product. It will tell your team if your product is performing in the competitive landscape.
- Make effective decisions. The usability metrics will inform your team as you make strategic decisions about the product and services.
Five usability metrics and how to measure them
Setting up usability metrics for your product or service is an easy task. But selecting the right metrics to get a proper result is more challenging. Here are five metrics designers can use to improve the design. These metrics will help you understand if you have achieved the three goals from the usability framework.
1: Task time
What it is: Task time measures the efficiency and productivity of the product or service. It is a core usability metric. If the user spends an extended amount of time completing an action, that means the interaction is not properly designed. Time is an important factor when considering the effectiveness of the solution. If the necessary functions are easy to find, then user satisfaction is high.
How to measure it: Designers can create scenarios for the users/focus groups to complete and track the time spent on completing the task. Use seconds and/or minutes to track the time. Ask the focus groups questions to identify problems they encounter.
Designers should record the following:
- Average time spent on task. This is the average time the focus group spends completing the task.
- Mean time to failure. This is the mean time the focus group spends on the particular task before giving up or completing the task with errors.
- Average task completion time. This is the average time it took for individuals to successfully complete the task.
Task time measures how long it takes to get the task done. Image credit Unsplash.Considerations: Task times are relative. If you are not conducting lab studies, there is a high possibility of external entities distracting the focus groups, preventing them from completing the task.
Example: Find a call-to-action button on a website. The designer can take a sample of 10 users and ask this focus group to find the location of the call-to-action button. Capture the time each participant spends on the task and then divide the time taken with the number of participants to find the average time taken to complete the work. If the average time taken to find the location is higher than the expected time, designers should focus on the information hierarchy of the application to see how it should be improved.
2: Errors
What it is: The errors usability metric helps designers understand where users get into trouble while trying to achieve their goals and, importantly, how they get into that trouble. Errors reduce user satisfaction and prevent users from coming back to the application. The number of errors a user makes during an interaction reveals how usable the application actually is to them.
How to measure it: Create a task for the user to carry out. Record all the mistakes, unintended actions, and slips that the user makes.
A application that generates few errors in interactions is considered a usable product. Image credit Unsplash.Considerations: If you can categorize the types of errors that users encounter, you’ll help your team focus on each pain point more effectively. Designers should always focus on preventing errors and detecting errors of the application.
Example: Assume a scenario where the user has to make an online bill payment. Simulate the actual scenario that the user has to go through in the application. Allow the user to conduct the task and accomplish the goal by successfully making an online bill payment. Observe how the user goes through the application; note any mistakes that they are making and how they are making those mistakes. If the user is making a huge number of errors while completing the task, the designer should look into the usability issues of the application carefully.
3: Completion rate
What it is: The completion rate is a critical, straightforward usability metric that can help to improve the usability of your product or service. The data collected is easy to understand because it is pass/fail—either users complete the task, or they don’t.
How to measure it: Create tests that have pre-defined success criteria. To get the most out of the results, conduct the tests in wireframe development.
A high completion rate means user satisfaction is high. Image credit Unsplash.Considerations: The context of use plays a valuable role in completion rate. Users expect high completion rates when working with an application that has a high consequence of failure. The completion rate will give you a general benchmark on how well the product is doing in achieving its intended goals.
4: Usability problems
What it is: This usability metric focuses on usability problems that occur at any phase of the interaction process. These issues can be identified when users:
- Elect the wrong links
- Misinterpret content
- Express frustration
- Miss out on the intended targets
- Don’t accomplish the task because of an application behavior
- Have an inaccurate sense of task completion
How to measure it: Identify how many usability problems occur in each user testing group, as well as the user group that encounters the highest number of problems. Conduct interviews with the users and ask them why they are encountering this problem to gain a clear insight into the usability problem at hand.
Usability is the key to create a delightful experience. Image credit Unsplash.Considerations: Usability problems occur in different user groups for different scenarios. Designers should keep a close eye on all the usability issues that occur during testing and follow up on the points of the application that caused users trouble.
5: Task satisfaction
What it is: The task satisfaction metric allows users to give feedback on the usability of the product, whether they have successfully achieved the task or not.
How to measure it: Once the user has finished, ask a few questions regarding the difficulty of the task. Ensure they are familiar with the questions.
Task satisfaction retains users. Image credit Dribbble.Considerations: Since designers can’t interview hundreds of users at a time, a solution is to create screens to gather customer feedback at the end of the interaction. This approach is most effective for consumer-based products. Offer levels of satisfaction as follows:
- Very dissatisfied
- Dissatisfied
- Neutral
- Satisfied
- Very satisfied
Allow each user to select a satisfaction level in the application. This will offer insight into emotions of focus groups on product experience.
Compare two designs
Comparing two different designs during usability testing will help you identify the qualitative results of the product or service and improve the overall experience of the application. To do this, select a particular application, create two designs of it to be tested, set up a few goal-oriented tasks for users to complete, and compare the results of the original and second versions. If results show that the completion rate of the redesign is faster than the original application, that means the user has performed tasks better in the redesigned version. But take caution not to over-test with multiple designs: It is really difficult to achieve a state where the user can perform a task within seconds.
Summarize the results
Designers should always focus on improving the usability of the application through a systematic process. Each usability testing process should conclude with a summary of the measurements and data gathered from the metrics in the tests performed. Designers should examine the relative importance of performance versus satisfaction of the application.
Designs should always be incrementally improved upon to gain maximum usability. Further A/B design testing could help designers more deeply understand the issues in the flow of the application and the aesthetic aspects of the applications as well. Summarizing results helps designers understand which design better works for the problem in hand.
Conclusion
Measuring usability will help designers understand the numerous improvements that need to be pursued. By implementing qualitative usability metrics and the usability framework throughout your testing, you’ll better understand what needs to be done to create an emotionally engaging application.
Data can’t tell us everything that is happening around us. Designers should always look into qualitative and quantitative measures to gather important data to validate usability. The most important question designers should always ask is not “How?” but “Why?” The usability framework presents a great approach to the question of why and helps designers tackle important areas of design.