Finally, you analyze the data you collected, creating a bunch of representations of the work in all its similarity and variety.
Actually, there are three levels of analysis, and only the last one is final. The first level of analysis occurs during the interview, as you observe and converse with the participant. This gives you the opportunity to get instantaneous feedback on your analysis and get your evolving model of the work checked against reality. You need to analyze the data from each interview as soon as possible after the interview has been conducted. Within twenty-four hours is the accepted time frame (i.e. the next day). Beyond that, you can expect the quality of the analaysis will deteriorate. There are two types of after-action analysis to be done. An individual one for each interview, and one that integrates the data across all the interviews. Analysis is done in a group with the interviewer presenting the data and the others discussing it and producing the products of analysis. The interviewer acts as informant, and "replays" the interview and the rest of the group (containing at least the rest of the site team and possibly others) discusses what is going on.
The interpretation session for each interview produces the following:
|
Database |
Contains all "observations, insights, design ideas, and questions" generated in the IS. [Example] |
|
Context Model |
Graphically depicts the socio-cultural influences on a work role. [Example] |
|
Physical Model |
Graphically shows the environmental context for the work. [Example] |
|
Flow Model |
Graphically represents the information producer/consumer relationship between the work role being studied and those it interacts with. [Example] |
|
Sequence Models |
Diagram the specific actions taken in time to accomplish a task. [Example] |
|
Artifact Models |
Representations of the artifacts the users were observed to use in carrying out their work annotated with the significance of its features (make sure to call out the subtle ones). [Example] |
The time expansion factor for the interpretation sessions is about 1.5, so expect to spend about six hours covering a four hour interview.
Once the interviews are complete and have been individually analyzed, it is time perform the summary analysis. In this phase you structure all the observations from all the interviews using an Affintity diagram, and you produce consolidated versions of the work models.
The Affinity diagram consists of all the entries from the database covering all the interviews. At 75 items per interview, and twenty interviews, this implies about 1500 data points you need to organize. You should have one person per 100 notes when building the diagram. The Affinity diagramming processs involves clustering items you feel are the same, or somehow strongly related together. There is a certain amount of shuffling to expected early on, but the groupings should eventually stabilize. At that point, you can think about categorizing the groupings. Avoid the temptation to come up with facile titles for still emerging groupings early on as it may divert your thinking into some preconceived rut instead of allowing the data to tell the real story. You will want to generate a hierarchical set of groupings with first level groups having not more than about a half-dozen items.
To consolidate the work models, you create versions of context, flow, physical, sequence, and artifact models that represent the commonality found across all the interviews and that note important variations. I'm limited to repeating Holtzblatt and Beyer's advice as there are no detailed descriptions of how this is done.
One thing you will find helpful in doing the analysis is having your own dedicated war room. This allows you to post your models on the wall, which allows easier access as well as a visual indication of the study's progress. It is also helpful to be able to leave your works in progress in place, instead of having to be putting them up and taking them down. It is also helpful to have a sizable wall to work on when creating your consolidated Affinity diagram.