A video record of the usability test can be extremely useful during and after the test has taken place. During the test it frees the experimenter from busily recording the information manually. This allows the experimenter to focus primarily on guiding the participants in the test, encouraging them to think aloud, and pose and answer questions regarding the usability of the product. It also eliminates the need for a seperate scribe and visiters to be present and possibly disturb the test. After the test has taken place, the videotape can be used to transcribe recorded information at a pace suitable to the scribe.
From the test, the company gets a report describing the usability problems of the system as well as its good points and a video record serves as a future reference to clarify or confirm issues or problems. Quite often, some ideas for new solutions are also introduced. In addition to the report, a feedback session with the designers can be arranged. The video record can also be extremely helpful in convincing higher level management and product designers that the problems with the product actually took place, along with identifing the severity level of each problem (Kennedy, 1989). Once a recording system has been set up, it is easy to maintain and adds to the importance and visibility of the usability test process, as the frequency of tests increases.
In thinking aloud the test user is asked to voice out their actions, thoughts and concerns regarding the product as they are using it in completing a task. It is critical that this information is collected while the test is taking place, rather than at the end, because the test user may forget some of the more subtle differences and may focus on the more difficult aspects of the product. If participants are provided with some examples and practice on thinking aloud then a desired level of information outflow can be attained. The thinking aloud technique can be helpful for some participants as it enables them to focus and concentrate on the tasks that they are performing. At times the participants will become quiet as they struggle in use of the product or simply will forget to think aloud. Then the experimenter must encourage thinking aloud and might ask general questions to re-orient the user to voice their actions, thoughts and concerns. Thinking aloud is however limited as it slows down the thought process and increases an awareness of what is being thought, which can sometimes hide problems that would have taken place with more routine thoughts and actions.
Even though the thinking aloud technique is extremely useful, some of the critical usability problems might remain hidden with the test being with one individual at a time. Situations can arise when the participants are unsure of a feature, but fail to voice their concerns, or when individual participants find it too difficult or unnatural to think aloud. In such cases it is better to test with two participants at a time, encouraging them both to think aloud and discuss the use of the product. This techique has been termed co-discovery learning (Kennedy, 1989) and constructive interaction (O' Malley et al., 1984), and can be more expensive and difficult to run as the experimenter has to watch and record two test users working together as well as with the product.
The pre test questionnaire can assess the participant's knowledge in the product's domain and experience with related products. While the post test questionnaire and interview provide overall impressions regarding the product and allow the experimenter to gauge the degree of importance of each feature. In the post test interview, the experimenter can also ask questions about possible problems that the test user faced while interacting with the product, follow up on some of the comments from the thinking aloud protocol and the questionnaires, as well as clear up some of the discrepancies between responses on the post test questionnaire and actual test occurances.