INTRODUCTION #3

What makes Usability Testing Different?

Some people will argue that user testing is already being performed in the traditional process by completing beta testing and academic research.

Usability Vs Beta testing: Beta testing is when a finished product is released early to certain groups of users as a trial period. The company supplying the product will contact the users to see how they like it after this trial period and possibly during the trial. There are no testers running the trial so the product may actually never get used, the regulation of use is up to the users. When users are asked they report what they remember and choose to report. When users do report something they may not be able to report the actions they took that resulted in the problem. Another characteristic of beta testing is the tasks that are tested are whatever users happen to do (i.e. some tasks may never arise, therefore untested). The major difference is that beta testing happens to late in the design process so that it is much more difficult to fix any of the problem(s) identified by the users (Dumas and Redish, 1993).

Usabilitiy Vs Research Studies: These two methods have many similarities (conduct tests in a lab, sample representative participants, controlling variables, recording measures, and analyzing the data). They have different goals; research wants to verify the existence of a theory whereas usability testing attempt to identify current problem areas. Each selects participants differently; usability testing selects users from a sample pool, and is not referred to as "scientific sampling". They have different views on controlling variables; a usability test does not usually isolate specific variables and tries to reflect real tasks in a real environment. They weigh observations differently; usability test teams' observations and comments are often given more weight in identifying the problem areas than they are in a research study. The major similarity between the two methods is the physical setting they take place (for the most part, a laboratory) and the major difference is the purpose they serve (Dumas and Redish, 1993).

Real users. Those testing the product are representative of the user population. The benefit of real users is they allows the developers to understand know their specific needs which can help in developing a better product.

Real tasks. The test tasks must be representative of how the product will be used in the "real" world. This will provide information on areas of the product that need to be changed/improved.

Observing and recording. Observations are made on participants' performance, comments/questions they make, and their behaviour. Observing and recording participants' behaviors distinguishes a usability test from focus groups, surveys, and beta testing.

Converting data into recommendations. The data collected (quantitative and qualitative data from participants together with observations and users' comments) can be analyzed and converted into problem areas. This allows problems to be prioritized and given possible solutions.

Changing the product and the process. When applied appropriately the results of a usability test will help improve the product and the development process. It is also a way to stimulate interest in usability within an organization and helps change people's attitudes about users. The impact of watching a few people struggle with a product will have more of an effect on attitudes than many hours of discussion. The ultimate organizational effect would be to change the design and development process that an organization uses to design and develop products (Dumas and Redis, 1993)..