Results from heuristic evaluations

Results from a heuristic evaluation consist of a list of usability problems with references to those usability principles that were violated by the design.  Evaluators should try and be specific and should list each usability problem separately.  There are two main reasons to note each problem separately [13]:

  1. even if an interface is completely replaced with a new design, there is risk of repeating problems unless one is aware of all its problems; and
  2. it may not be possible to fix all usability problems in an interface or to replace it with a new design, but it could still be possible to fix some of the problems if they are all known.

Studies [11] have shown that heuristic evaluations tend to find more minor problems than major.  However, the probability of finding major problems is greater than finding minor ones.

It could be expected that usability problems violating certain heuristics would be easier to find than others.  Nielsen [11] shows the effectiveness of finding usability problems using the 9 of the general heuristics (excluding "help and documentation").  It was discovered that user interface problems identified during a heuristic evaluation performed by three evaluators will have about the same probability of being found for most of the heuristics.  However, problems that violated the "Prevent errors" and "Clearly marked exits" heuristics were seemingly more difficult to identify.  It was recommended that additional measures should be taken to find problems relating to these heuristics.

In another study, Nielsen [14] determined that the majority of usability problems identified in a system were not due to a few factors.  There was a gradual decline in the number of usability problems that violated each heuristic.  In other words, usability problems were due to a broad variety of heuristics being violated.

Nielsen [11] also looked at whether the circumstances under which the problems could be located had any influence.  He stated that usability problems can be located in four different ways:

  1. in a single dialogue element;
  2. in two or more locations that have to be compared to find the problem;
  3. as a problem with the overall structure of the interface; and
  4. as something that is missing and should be included.

It was found that evaluators were just as good at finding usability problems no matter what kind it was.

Return to Home Page