Heuristic evaluation is a usability research method that consists in inspecting a UI in order to find usability issues by checking it against a set of recognized heuristics ( = industry-standard usability principles). It is a peculiar usability method because it does not involve end users: to evaluate the interface is usually a panel of industry “experts” (designers, engineers, etc.).
What is a heuristic evaluation
In other words, in a heuristic evaluation a group of expert evaluators will each individually review the interface using as a reference a set of industry-standard guidelines for usability. Each evaluator will then identify a list of usability issues and weaknesses in the UI. A study from Nielsen Norman Group shows that a panel with three-five evaluators is sufficient to cover a majority of usability issues, although more are recommended for evaluating critical systems. This is because even though heuristics can save time and help uncover issues before testing with users, the use of experts can be expensive. And because this is not a user test, it cannot be used to investigate the actual user experience: testing with experts – who already possess knowledge of the product and field being evaluated – cannot replace testing with the target end users.
There is not a single set of heuristics that can be used to conduct heuristic evaluation. Each type of product and interface has their own specific design principles which are still being improved upon (this is particularly true if you are working with new technologies like voice interfaces, virtual reality, etc.).
To give you an example, the most well-known heuristics for generic usability research are Nielsen Norman Group’s
Usability Heuristics for Interface Design:
#1: Visibility of system status
The design should always keep users informed about what is going on, through appropriate feedback within a reasonable amount of time.
#2: Match between system and the real world
The design should speak the users’ language. Use words, phrases, and concepts familiar to the user, rather than internal jargon. Follow real-world conventions, making information appear in a natural and logical order.
#3: User control and freedom
Users often perform actions by mistake. They need a clearly marked “emergency exit” to leave the unwanted action without having to go through an extended process.
#4: Consistency and standards
Users should not have to wonder whether different words, situations, or actions mean the same thing. Follow platform and industry conventions.
#6: Recognition rather than recall
Minimize the user’s memory load by making elements, actions, and options visible. The user should not have to remember information from one part of the interface to another. Information required to use the design (e.g. field labels or menu items) should be visible or easily retrievable when needed.
#7: Flexibility and efficiency of use
Shortcuts — hidden from novice users — may speed up the interaction for the expert user such that the design can cater to both inexperienced and experienced users. Allow users to tailor frequent actions.
#8: Aesthetic and minimalist design
Interfaces should not contain information which is irrelevant or rarely needed. Every extra unit of information in an interface competes with the relevant units of information and diminishes their relative visibility.
#9: Help users recognize, diagnose, and recover from errors
Error messages should be expressed in plain language (no error codes), precisely indicate the problem, and constructively suggest a solution.
#10: Help and documentation
It’s best if the system doesn’t need any additional explanation. However, it may be necessary to provide documentation to help users understand how to complete their tasks.
Source: Nielsen Norman Group
How to conduct a heuristic evaluation
When conducting a heuristic evaluation, you will want the evaluators to be well familiar with the chosen set of heuristics. I usually print them out together with a template that they can use to list down any usability issues, personal notes and severity ratings for issues they find.
A severity rating would be a score from 0 to 4, as following:
- 0 = I don’t agree that this is a usability problem at all
- 1 = Cosmetic problem only: need not be fixed unless extra time is available on project
- 2 = Minor usability problem: fixing this should be given low priority
- 3 = Major usability problem: important to fix, so should be given high priority
- 4 = Usability catastrophe: imperative to fix this before product can be released
While there is not an entirely objective way to assess the severity of usability issues, it usually comes to a combination of frequency of the issue (how often does it appear?), its impact on the user experience (does it slow down the user or prevent them from performing a task?) and its persistence (can the user find a workaround or ignore the issue and still perform their task?). In this regard, a cosmetic problem is purely aesthetic, but has minimal impact on the UX, whereas a usability catastrophe prevents the user from carrying out their task altogether.
Evalution and debriefing
During each individual session, the evaluator will go through the interface, locate usability issues, and give them a rating. At the end, I like to have a debriefing session where all the evaluators sit together and go through their feedback, discussing what they found with each other. The outcome of the session should be a final list of usability issues ranked by severity rating – it will ultimately be up to you, as the facilitator, to decide on their severity based on everyone else’s feedback. Ideally, you should come up with an action plan for addressing the top priority items.