Heuristic Evaluations

Formal Evaluations

Tutorial

Heuristic evaluation is a tool that can help us catch usability and design flaws in a user interface. It is performed by a small set of evaluators. (3-5, likely 3 for our case: 1 Author, 2 Reviewers) This evaluation can be performed on both working UI and sketches.

How does it Work?

• First choose what is being evaluated, e.g. the entire UI, individual user tasks, etc...
  • A task might be something like: "create a new message starting from the home page".
  • Tasks should be plausible actions a user might perform -- may correspond to our functional requirements
• Evaluators go through the UI multiple times and independantly check for consistency with "heuristics"
  • Any inconsistencies are recorded and assigned an estimated severity rating
• Evaluators then meet together and aggregate their results

Nielsen’s 10 Heuristics

Ref: 10 Usability Heuristics for User Interface Design

Listed below are Nielsen’s revised heuristics:

• H2-1 Visibility of System Status
• H2-2 Match between system and the real world
• H2-3 User control and freedom
• H2-4 Consistency and standards
• H2-5 Error prevention
• H2-6 Recognition rather than recall
• H2-7 Flexibility and efficiency of use
• H2-8 Aesthetic and minimalist design
• H2-9 Help users recognize, diagnose and recover from errors
• H2-10 Help and documentation

Severity Rating

Ref: Severity Ratings for Usability Problems

The severity of a usability problem is a combination of 3 factors:

• frequency -- how often it occurs
• impact: -- how easy it is for users to overcome
• persistence -- will it repeatedly bother users

The following scale can be used to rate severity:

• 0 = No usability problem
• 1 = Cosmetic problem: Time permitting
• 2 = Minor usability problem: Low priority
• 3 = Major usability problem: High priority
• 4 = Usability catastrophe: Immediate priority

Example Evaluation

Problem	Heuristic	Eval 1	Eval 2	Eval 3	Final Rating
No "back" button in menu	H2-3	2		3	3

The example above shows that the given UI problem was discovered by 2 of the 3 evaluators to violate the H2-3 heuristic. The final rating corresponds to the mean of the individual evaluation ratings rounded up to the nearest whole number.