1) Purpose statement. This is a high level reason for performing the test. This should be the result of some issue that has been raised by user input or change in policy that all products must be test. A desire to have a usability test just because everyone else is doing it is not appropriate. An example of the a proper purpose statement is :
2) Problem statements / Test objectives. These statements are most important in defining the issues and questions to be resolved as well as finalizing the type of usability test to use. These should be not be vague but specific. An example of a proper problem statement is :
3) There must be a decision made on the goal of the usability test. If the test goal is formative then one of the four formative usability type tests can be persued. If the test is goal is summative then a controlled experiment can be undertaken. This decision process will be relatively easy to make, dependent on the present design phase, but it is critical that a clear statement be made as it is extremely difficult to shift gears from a design iteration formative test to a more controlled summative test.
Formative testing can be exploratory, assessment oriented, validation oriented or comparative. A decision on the type of formative test is also important, and highly dependent on the current design phase and circumstances.
4) Development of a User Profile.
5) Method of test. This detailed description is how the research is going to be carried out with the participants, which methodological design, either a between or within subjects, will be used. This description would include a list of the features of the test, along with how the feature tests are going to be randomized for each individual if testing using a within subject design. If the test is performed with two individuals, as in co-discovery learning, then the need and use of this technique would also be made clear in this description.
6) Task List. A representative list of tasks to be performed by the users is essential to maintain a high degree of professionalism during the test and avioding confusion and headaches for both the test experimenter as well as the participant. The tasks in this list should consist of tasks that will ordinarily be performed by the eventual user. These tasks can be rated on frequency of use and degree of importance. A detailed description should be made as to which actions or behaviours are required for each successful task. These actions should be as specific as possible.
7) Test Environment. A location for the usability test should be chosen. This area should be relatively quiet and comfortable. This area should be equiped with a desk and chairs for the participant, the experimenter and the product. A video recording mechanism is recommended but is not required for testing. It can however be extremely beneficial to record the proceeding especially when a high degree of thinking aloud information is to be collected and when revisiting a specific test can be helpful at later times.
8) Evaluation Measures. Both performance and preference data can be collected in usability tests. The choice of measure is dependent on the problem statement and objective of the usability test. The following is a list of possible performance and preference data measures.
Performance
Preference
Rubin (1994) provides excellent examples of the requirements of the test plan in his book, Handbook of Usability Testing: How to Plan, Design, and Conduct Effective Tests. Wiley: NY.