In terms of dollars and time spent, the cost of a usability test depends on the size and complexity of the product, the scope of the product's development project and the number of participants per usability test.
If the product is relatively early in the development phase, the product is in the paper design phase and the designers are testing the usability within their own group then the test can be performed rather quickly and cheaply. For such early experimentation and for most usability tests that are performed for which no formal usability program has been set up, a simple set up consisting of a desk, the product, a video recorder and questionnaire forms are sufficient. With the use of co-discovery learning, two participants are required at a time to interact with the product, and this increases the renumeration to be paid.
As the product nears completion or increases in complexity then a more formal testing mechanism would be required with proper recruitment of representative participants and increased costs. There is a also an increased cost in time spent in setting up tasks, familirizing the experimenter with the product, the tasks and the experimental set up, and obtaining ethical approval from an established ethics board. It is critical that all ethics procedures be adhered for both early paper prototype testing and during later product prototype testing.
Nielsen (1993) states that estimates for a representative, medium-sized usability test can be derived, with fixed costs of $3,000 and variable costs of $1,000 per test user. I feel that this is a really high estimate and would probably be true for more formalized tests in larger companies with lots of money to spend. In our usability test on overviews, we are hoping to test with the least amount of cost in terms of dollars.
It is however recommended that usability testing be performed rather early in the design phase with paper prototypes as modifications to the design are relatively easy to implement during early phases rather than later phases when the product is nearing completion. If such an approach is used, then the cost of the usability test is relatively low and the benefits gained from the participant comments, if implemented, can reduce some of the major usability problems with the final product.
Usability testing during early phases can thus be "quick and dirty .. on the cheap" (Dr. Saul Greenberg)
How many test users are sufficient in finding out the main issues ?
Approximately 80 percent of the usability problems in a product can be detected with 6 test users per test grouping. As the number of test users increases beyond 6, there is a dimishing returns factor, as most of the major usability problems have already been revealed (Nielson, 1993). With a large percent of usablity problems being identified with six participants for each iteration of the product, the process of usability testing is internally reliable and valid. The cost of running an experiment increases with more participants, and thus there needs to be a balance between time and money spent, and information gained.