Heuristic Evaluation of Groupware

Many organizations, particularly in the design and manufacturing sectors, are becoming increasingly distributed. Work teams are often made up of people located at different sites, and experts are rarely present in the same location. As a result, the use of collaboration technology is fast becoming a necessity in these organizations, and the usability of the technology can have a large impact both on the overall efficiency and effectiveness of the team, and on the quality of the work that they do. However, there is currently no good definition of groupware usability, and little known about how to evaluate it. In this project with University of Saskatchewan (see http://hci.usask.ca/projects/gwevaluation.xml, we are investigating the problem of groupware usability and exploring means and methods for evaluation of groupware.

We are adapting the heuristic evaluation inspection technique developed for conventional discount usability evaluation to groupware. In particular, we developed one set of heuristics from the Locales framework for group interaction, and another set based upon the low level 'mechanics' of collaboration. We then used these new heuristics to inspect various groupware interfaces, and we were able to identify both strengths and weaknesses of these systems.

Heuristics (Locales Framework)

  • Provides Centers (locales) that collect people, artifacts and resources in relation to the central purpose of the social world. A locale provides the site, means and resources for a group to pursue team and task work. Locales should be dynamic so they can evolve along with the people, the artifacts, and the purposes that defines them.
  • Provide Awareness (mutuality) within Locales that help people maintain a sense of shared place and that keeps them informed about shared activity. Mutuality includes one person's awareness of others, the artifacts comprising the locale, where things are located, and how things are changing.
  • Allow Individual Views so one can view a locale or aggregate multiple locales as they relate to one's responsibilities, activities, and interests. A particular person should be able to view locales from his or her particular perspective and in a way that reflects their degree of focus and participation.
  • Allow People to Manage and Stay Aware of Their Evolving Interactions Over Time. This includes a group's control over past, present and future aspects of routine and non-routine work; how people coordinate and negotiate plans and activities over time; how people leverage past experiences; how breakdowns are noticed and repaired; and how processes are supported.
  • Provide a Way to Organize and relate Locales to One Another (civic structures). Locales are rarely independent of one another: people need a way to structure the locales in a meanignful way, to find their way between locales, to create new locales, and to remove old ones.

Primary Investigators

Kevin Baker (MSc Project)
Saul Greenberg (Supervisor)
Carl Gutwin (Supervisor)

Milestones

  • Grant received from NIST to fund this work.
  • Two sets of heuristics developed.

Current Status

  • We are applying heuristic evaluation methods to groupware.

Key Papers

  • Baker, K., Greenberg, S. and Gutwin, C. (2000) Heuristic Evaluation of Groupware Based on the Mechanics of Collaboration. Report 2000-669-21, Department of Computer Science, University of Calgary, Alberta, Canada., October. [Pdf]
  • Greenberg, S., Fitzpatrick, G., Gutwin, C. and Kaplan, S. (2000) Adapting the Locales Framework for Heuristic Evaluation of Groupware. Australian Journal of Information Systems (AJIS) 7(2), 102-108, May. (Reprinted from Proceedings of the OZCHI'99 Australian Conference on Human Computer Interaction, November 28-30, Wagga Wagga, Australia).