In the article, "Getting the Whole Team into Usability Testing," Kate Ehrlich, Mary Beth Butler, and Kara Pernice of the Lotus Development Corp. describe four approaches they found to be valuable in getting non-usability people (i.e., other team members and developers) involved in usability testing. Although described as a tool for getting developers and designers involved, it can also be applied to getting upper management individuals involved. The methods are summarized below:
This method involves the developer/designer observing usability testing performed at an informal level. Typical scenarios involve testing in someone's office on their computer. The usability engineer sits next to the user and the interface designer or developer sits by and observes. The informality of the situation tends to make it easy for the designer/developer to witness user difficulties and ask direct questions to the user. In summary, the informal setup is an effective way to expose people first hand to usability testing (the effectiveness of seeing user difficulties with one's own eyes are well known) without investing too many resources. On the down side, however, this method tends to create a lot of "setting-up" work for the usability engineer and as such is not recommended as a long term solution.
Similar to the premise of the informal setup method, this method involves allowing developers/designers to observe usability testing, this time in the formal setting of a usability lab. Observers are located in an observation room which provides some physical separation to the subject testing room. The advantages of the sit-in sessions mirror the those of the informal setups - the developers and designers could see first-hand the difficulties experienced by users. Additionally, several team members can witness a test session simultaneously and the fact that the group sees the usability tests together facilitates very effective communication between usability engineers, interface designers and developers. The main disadvantage of this method is the requirement that developers and designers attend at least most of the test sessions (a difficult thing for those who are busy). Additionally, there is the tendency of non-usability professionals to misinterpret user behaviours or miss subtle problems.
During "usability nights," 10 to 12 users came to Lotus after working hours where each one was paired with one or two members of the development team. The developers then spent about two hours observing the user work on the software being tested. The developers were instructed to ask questions about how the user worked and take notes while observing the users, but to minimize any assistance to the user. Usability staff then consolidated the developers notes into a final report. Like the other methods, this method provides team members with hands-on experience in dealing with actual users and seeing user problems first hand. Additionally, usability nights lifted morale significantly. The fact that there was a roomful of excited users and developers all working toward improving the product went a long way in keeping team members focused on usability. The major disadvantage is the lower quality of data-gathering since developers are not generally trained in user observation and note taking.
To effectively utilize videotapes, they should be edited to summarize and highlight the main usability problems of a product. This is much less time consuming for developers and still preserves the effectiveness of seeing user difficulties first-hand. On the other hand, videotape preparation itself is a time-consuming process.
Return to Grassroots Acceptance