A Comparison of Usage Evaluation and Inspection Methods for Assessing Groupware Usability

Steves, M.P., Morse, E., Gutwin, C. and Greenberg, S. (2001)
A Comparison of Usage Evaluation and Inspection Methods for Assessing Groupware Usability. In Proceedings of ACM International Conference on Supporting Group Work - ACM Group'01. ACM Press, pages 125-134, September 30-October 3.

View Publication and Related Materials

PDF PaperPDF Paper (2001-GWMethodologyComparison.Group.pdf)

Abstract

Many researchers believe that groupware can only be evaluated by studying real collaborators in their real contexts, a process that tends to be expensive and time-consuming. Others believe that it is more practical to evaluate groupware through usability inspection methods. Deciding between these two approaches is difficult, because it is unclear how they compare in a real evaluation situation. To address this problem, we carried out a dual evaluation of a groupware system, with one evaluation applying user-based techniques, and the other using inspection methods. We compared the results from the two evaluations and concluded that, while the two methods have their own strengths, weaknesses, and trade-offs, they are complementary. Because the two methods found overlapping problems, we expect that they can be used in tandem to good effect, e.g., applying the discount method prior to a field study, with the expectation that the system deployed in the more expensive field study has a better chance of doing well because some pertinent usability problems will have already been addressed.

Bibtex entry

@INPROCEEDINGS { 2001-GWMethodologyComparison.Group,
CLASS = { CONFARTICLE },
AUTHOR = { Steves, M.P. and Morse, E. and Gutwin, C. and Greenberg, S. },
TITLE = { A Comparison of Usage Evaluation and Inspection Methods for Assessing Groupware Usability },
BOOKTITLE = { Proceedings of ACM International Conference on Supporting Group Work - ACM Group'01 },
PAGES = { 125-134 },
YEAR = { 2001 },
MONTH = { September 30-October 3 },
PUBLISHER = { ACM Press },
}