Despite the name, heuristic evaluations are not as complex as they sound. In fact, they’re a great tool for teams with various budgets for testing, as they can be executed in a couple of different ways.
What is a heuristic evaluation?
A heuristic technique is an approach of discovery or problem solving that has broad guidelines, or rules of thumb, without any rules etched in stone. It’s a guided look into a problem or study.
As it applies to UX and product design, a heuristic evaluation is a method of inspecting and evaluating the usability of a website, or product. You may also hear it referred to as a “usability audit” or an “expert review”. Using a set of heuristics, one or more experts will evaluate how well a product complies to these heuristics to define its usability. Often they’ll leverage a score-card, or numeric-based scoring (weighted to the impact on usability) for each heuristic analysis.
The value of heuristic evaluations is prevalent in early stage design and development, and in smaller organizations which may not be equipped with the budget or resources to support a robust user-testing program but still need to validate design decisions, and ensure a good user experience.
How a heuristic evaluation is different from usability testing
Usability testing leverages users of a product, or people representing the intended target audience, to understand and identify areas of friction or error that may prevent completion of specified tasks, or lead to dissatisfaction in a target group. They may be seasoned users evaluating a new feature, or new users evaluating a particular user story. One primary difference is the resourcing required to run effective usability tests. It can become cumbersome to source users for a test, and once feedback is incorporated into an iteration, new users must be sourced to validate that solution too.
However, with a heuristic evaluation, you’re evaluating with 1-3 experts, who don’t need to be users of the product. If you can’t find an expert (a usability professional knowledgeable in these evaluations) you can perform something called a heuristic markup, as described by Leah Buley in her book The User Experience Team of One.
The heuristic markup approach is a less formal method that can be executed by your internal team, and involves navigating the product along the journey map you expect your users to experience. Make notes along the way and use a set of ‘heuristics’ to help guide you.
Helpful heuristics to follow
A heuristic is a broad set of standards — there are no official heuristics to use when conducting your evaluation. However, there are some very commonly used heuristics, such as those written by Jakob Nielsen of the Nielsen Norman Group. Released in 1994 in his book Usability Engineering the following 10 Heuristics are a staple in the field, and are considered a standard when evaluating usability.
- Visibility of system status. Users should always be informed of what the system, or product is doing, in a reasonable timeline.
- Match between system and the real world. Avoid technical jargon. The product interface should align to terminology and language familiar to the user.
- User control and freedom. Mistakes can happen, provide a way out when they do. Supporting undo and redo is one way of achieving this, along with methods of cancelling an operation.
- Consistency and standards. Don’t reinvent the wheel. Users understand conventions and patterns, and should not have to guess as to what something means. Follow patterns that exist across platforms.
- Error prevention. Providing feedback when an error occurs is important, but better yet if you can help prevent an error from occurring in the first place. Providing smart defaults, and confirmation is beneficial.
- Recognition rather than recall. To maximize usability, reduce the load on memory and recall. Avoid hiding important actions behind menus, and instead present options in a visible way so users don’t have to remember where they are.
- Flexibility and efficiency of use. Accommodate both novice and advanced users, but tailor the experience. Provide ways of speeding up workflows, and accelerating users familiar with the system, while guiding those less familiar.
- Aesthetic and minimalist design. Don’t overwhelm users by displaying unnecessary information. Keep screens and dialogues focused and minimal to maximize visibility and clarity.
- Help users recognize, diagnose, and recover from errors. Errors will occur, and it is important to help users understand what caused the error, in plain language, and how they can go about rectifying the situation.
- Help and documentation. Though it is best to design away the need for help and documentation, it is important to make it accessible when it is needed. Don’t make users struggle to find help, and where possible present it contextually as needed, in plain, clear language.
Other principles to consider when developing your heuristics are Gerhardt-Powals’ cognitive engineering principles from the International Journal of Human-Computer Interaction.
These heuristics provide a strong core, however, they were written in the early 90s, at a time when technology products, especially software, were in a very different state. Though for the most part, these hold true today, it may be beneficial to review these and reflect on whether they’re the right heuristics to use for your own particular product. Using these heuristics along with principles and guidelines specific to your product will make for more effective testing and better results.
Guidelines for effective heuristic evaluations
Choose evaluators
You want to make sure when selecting evaluators, that they are not end users of your product. However, some understanding of the industry or product space is helpful in evaluating the usability for that audience.
If you’re in a large company with the resources to hire external usability experts, many companies, like Nielson Norman group provide consulting services which include usability reviews and heuristic evaluations. Hiring an outside expert can provide a fresh set of eyes, and industry experience to provide valuable feedback on how to improve the usability of your site.
Provide direction
It is likely that evaluating your entire product or website will be a large task, and if you’re hiring experts, could become expensive. Ensure efficiency in your evaluation by providing direction, such as user stories, or particular tasks you wish the experts to evaluate. However, ensure that you provide the same criteria to all experts to minimize the risk of bias in results. Allow for some exploration beyond the user-stories, as there may be avenues the experts wish to explore based on their industry knowledge, and past experience.
Evaluate
For this step it is beneficial to have someone, familiar with the product, in the room to document feedback, and provide answers to any questions evaluators may have. Having a note-taker ensures you don’t miss any valuable feedback that may arise.
When evaluating, it is recommended that you do two reviews. The first, allowing the experts to get familiar with the product, and interactions. It is in this phase they may pinpoint additional areas they would like to evaluate.
In the second phase, the website or product will be evaluated against the defined heuristics to the user stories, and the additional elements identified for evaluation. This is where most of the feedback, and problem areas will arise.
Review
Once you’ve run your tests, it is helpful to bring the experts together to cross-examine the problems identified by each person, it is likely that there will be overlap between experts. Having more than three evaluators, you will begin to see a sharp reduction in returns, and so it is recommended to use no more than three experts.
It is likely that, based on previous experience, the experts may produce some solutions and recommendations based off of the recorded issues. Don’t prevent this from happening, it is valuable feedback that is likely to benefit your product. However, do evaluate all feedback before implementing solutions.
When to use heuristic evaluations, and when not to
Heuristic evaluations are not the be-all, end-all. They’re one of many usability testing methods UX designers have at their disposal. However, they are still guidelines, and some of the feedback you’ll receive may not actually make sense for your product.
Benefits:
- Heuristic evaluations can help to identify issues with particular components or sections of a product that have a negative impact on usability.
- They provide insight from experts, and can help identify usability problems early on in a design or development cycle.
Drawbacks:
- Heuristic evaluations rely heavily on the heuristics that are defined or chosen. If they’re misunderstood, you may receive biased results.
- They’re based on existing, global ideas of what makes something usable. However, this can change by product, or audience. Experts not versed in an industry may make assumptions that actually provide misleading feedback. That being said, this is rare, as well formed heuristics like those from Nielsen come from years of research and user testing experiences.
When implementing heuristic evaluations, understand the pros and cons, and be aware of false positives in reporting. It is always valuable to pair heuristic evaluations with some level of user-testing to ensure the problems identified by experts are in fact problems impacting your usability. Having multiple experts can help reduce false alarms, and ensure bias is reduced in your results.