Usability Evaluation Considered Harmful (Some of the Time)

Greenberg, S. and Buxton, B. (2008)
Usability Evaluation Considered Harmful (Some of the Time). In Proceedings of the ACM Conference on Human Factors in Computing Systems - ACM CHI'08. (Florence, Italy), ACM Press, pages 111-120, April 5-10. Best paper nomination. Earlier version as Report 2007-879-31. See also minute madness video.

View Publication and Related Materials

PDF PaperPDF Paper (2008-UsabilityHarmful.CHI.pdf)
PowerPoint PresentationPowerPoint Presentation (2008-UsabilityHarmful.CHI.ppt)

Abstract

Current practice in Human Computer Interaction as encouraged by educational institutes, academic review processes, and institutions with usability groups advocate usability evaluation as a critical part of every design process. This is for good reason: usability evaluation has a significant role to play when conditions warrant it. Yet evaluation can be ineffective and even harmful if naively done 'by rule' rather than 'by thought'. If done during early stage design, it can mute creative ideas that do not conform to current interface norms. If done to test radical innovations, the many interface issues that would likely arise from an immature technology can quash what could have been an inspired vision. If done to validate an academic prototype, it may incorrectly suggest a design's scientific worthiness rather than offer a meaningful critique of how it would be adopted and used in everyday practice. If done without regard to how cultures adopt technology over time, then today's reluctant reactions by users will forestall tomorrow's eager acceptance. The choice of evaluation methodology -- if any -- must arise from and be appropriate for the actual problem or research question under consideration.

Keywords

Usability testing, interface critiques, teaching usability, ACM: H5.2. Information interfaces and presentation (e.g., HCI):User Interfaces (Evaluation/Methodology)

Bibtex entry

@INPROCEEDINGS { 2008-UsabilityHarmful.CHI,
CLASS = { CONFARTICLE },
AUTHOR = { Greenberg, S. and Buxton, B. },
TITLE = { Usability Evaluation Considered Harmful (Some of the Time) },
BOOKTITLE = { Proceedings of the ACM Conference on Human Factors in Computing Systems - ACM CHI'08 },
YEAR = { 2008 },
MONTH = { April 5-10 },
PUBLISHER = { ACM Press },
ADDRESS = { Florence, Italy },
NOTE = { Best paper nomination. Earlier version as Report 2007-879-31. See also minute madness video },
PAGES = { 111-120 },
KEYWORDS = { Usability testing, interface critiques, teaching usability, ACM: H5.2. Information interfaces and presentation (e.g., HCI):User Interfaces (Evaluation/Methodology) },
}