When the development of the product is nearing its end, a representative sample of users should be involved in the assessment of the usability. In order to get a representative sample the designers, marketers and usability specialists can create a user profile on the characteristics that are relevant to the successful completion of the tasks. These user profiles can be developed through market surveys, focus groups, and by observing and interviewing perspective users. During pilot testing and testing in early development phases an available subset of internal expert users can test the product's use . These users should however be unfamiliar with the product before the test is to take place. Issues that the expert users face will also be faced by novice users. For example, in a usability test of transparent interfaces, if the expert users are experiencing visual interference in discriminating artefacts in the foreground view from the background view, then it can be assumed that novice users will experience the same interference.
When developing a product, such as interfaces, the designers should pay close attention to the level of expertise that the perspective users will have. If the interface is to be designed for individuals that display a range of computer expertise, novice to expert, then the group selected for usability testing should also exhibit the same range of computer experience. Computer expertise levels can be divided into use with a particular application or overall years of computer use. For example, some participants may have used computers for 10 years and might not be familiar with Macintosh computers. The experimenters should also consider if training in domain area is needed before testing the usability (Nielson, 1993). Care must be taken when testing with special groups of participants, for example when testing a new input device for children.
Approximately 80 percent of the usability problems in a product can be detected with 6 test users per test grouping. As the number of test users increases beyond 6, there is a dimishing returns factor, as most of the major usability problems have already been revealed (Nielson, 1993). With a large percent of usablity problems being identified with six participants for each iteration of the product, the process of usability testing is internally reliable and valid. Obviously the cost of running an experiment increases with more participants, and thus there needs to be a balance between time and money spent, and information gained.
When testing the usability of a product, with potential users, the experimenters should be unbiased towards the product itself. Designers of the product tend to grow attached to their creations, and sometimes are not accepting of the criticisms that the product may face during the test. Uncomfortable situations may rise if the participant is struggling with the product, and the designer attempts to interfere by providing suggestions on use. However, it is equally important that designers of the products become aware of the problems that the participants faced while using the product, as one of the benefits of the test is the barrier that exists between the designers who create the product and the individuals that use it. This can by video taping the usability test or by the designer being present in an adjacent room that has been separated by a one-way mirror. The experimenters should be familiar with the product, so that they can offer advice or assistance if the participant is clearly unable to use the product in any way. Usability test experimenters should also try to develop the ability to sense when the participant needs the help, is uncomfortable in the use or needs reassurance to keep thinking aloud or voicing the actions they are taking and thoughts that they are having.