In 1990, Jakob Nielsen and Rolf Molich adapted a psychological principle, called Heuristics, to user-interface design. Nielsen, in particular, has promoted his ten recommended heuristics as a part of heuristic evaluation. But what is heuristic evaluation? According to Nielsen, “Heuristic evaluation is a discount usability engineering method for quick, cheap, and easy evaluation of a user interface design.”

So, if heuristic evaluation is a method, then what is a heuristic? Nothing more (or less) than a rule of thumb. Essentially, heuristic evaluation utilizes a set of established principles to analyze the usability of an interactive system (a website, an application, a kiosk, etc.). Nielsen has published his ten usability heuristics on the useit.com website, and they remain generally appropriate; the system should keep the user informed of what is going on, language should be natural and based on real-world conventions,navigation systems should be consistent, the system should be flexible and efficient.

All good ideas, in a general sense. But in a specific context, they can be (and often are) misused and misapplied. Most often, heuristic evaluation is conducted by design “experts,” which means that the experts bring their personal bias, history, and preferences into the game. Yes, the idea of this type of analysis is to remove personal preference from the process, but in real life this is easier said than done. A design solution appropriately utilized in one situation can be wrong in another.

The potential for errors in heuristic evaluation was explored effectively in an article I read in The New Yorker last night; the article, entitled “What’s the Trouble? How Doctors Think” was written by Jerome Groopman, a Professor of Medicine at Harvard. To quote Dr. Groopman:

But research shows that most physicians already have in mind two or three possible diagnoses within minutes of meeting a patient, and that they tend to develop their hunches from very incomplete information. To make diagnoses, most doctors rely on shortcuts and rules of thumb—known in psychology as “heuristics.”

Heuristics are indispensable in medicine; physicians, particularly in emergency rooms, must often make quick judgments about how to treat a patient, on the basis of a few, potentially serious symptoms. A doctor is trained to assume, for example, that a patient suffering from a high fever and sharp pain in the lower right side of the abdomen could be suffering from appendicitis; he immediately sends the patient for X-rays and contacts the surgeon on call. But, just as heuristics can help doctors save lives, they can also lead them to make grave errors.

Dr. Groopman goes on to list several types of errors made in heuristic analysis, including “representativeness”, “availability”, and “affective” errors.

Representativeness errors are made by doctors when “their thinking is overly influenced by what is typically true; they fail to consider possibilities that contradict their mental templates”.

He describes availability errors in the following manner:

[…] the tendency to judge the likelihood of an event by the ease with which relevant examples come to mind. This tendency was first described in 1973, in a paper by Amos Tversky and Daniel Kahneman, psychologists at the Hebrew University of Jerusalem. For example, a businessman may estimate the likelihood that a given venture could fail by recalling difficulties that his associates had encountered in the marketplace, rather than by relying on all the data available to him about the venture; the experiences most familiar to him can bias his assessment of the chances for success. (Kahneman won the Nobel Prize in Economics in 2002, for his research on decision-making under conditions of uncertainty.)

If representativeness and availability are intellectual errors, affective errors are those caused by an emotional connection. As described by Dr. Groopman, affective errors are “errors that doctor make because of their feelings for a patient.”

All of these errors can be associated with work in interaction design as well as in medicine. To use heuristic evaluation effectively, we all must understand the way our minds work. Consider the following quote, and replace the doctor with the interaction design expert, and the starting point for addressing the limitations of heuristic evaluation is clear:

This approach produces confident and able physicians. Yet the ideal it implies, of the doctor as a dispassionate and rational actor, is misguided. As Tversky and Kahneman and other cognitive psychologists have shown, when people are confronted with uncertainty—the situation of every doctor attempting to diagnose a patient—they are susceptible to unconscious emotions and personal biases, and are more likely to make cognitive error. […]the first step toward incorporating an awareness of heuristics and their liabilities into medical practice is to recognize that how doctors think can affect their success as much as how much they know, or how much experience they have.