Prototyping an Intelligent Agent through Wizard of Oz

Maulsby, D., Greenberg, S. and Mander, R. (1993)
Prototyping an Intelligent Agent through Wizard of Oz. In Proceedings of the ACM Conference on Human Factors in Computing Systems - ACM CHI'93. (Amsterdam, The Netherlands), ACM Press, pages 277-284, May. Also as Report 1992-489-27, September.

View Publication and Related Materials

PDF PaperPDF Paper (1993-Woz.CHI.pdf)

Abstract

Turvy is a simulated prototype of an instructible agent. The user teaches it by demonstrating actions and pointing at or talking about relevant data. We formalized our assumptions about what could be implemented, then used the Wizard of Oz to flesh out a design and observe users' reactions as they taught several editing tasks. We found: a) all users invent a similar set of commands to teach the agent; b) users learn the agent's language by copying its speech; c) users teach simple tasks with ease and complex ones with reasonable effort; and d) agents cannot expect users to point to or identify critical features without prompting.

In conducting this rather complex simulation, we learned some lessons about using the Wizard of Oz to prototype in-telligent agents: a) design of the simulation benefits greatly from prior implementation experience; b) the agent's behavior and dialog capabilities must be based on formal models; c) studies of verbal discourse lead directly to an implementable system; d) the designer benefits greatly by becoming the Wizard; and e) qualitative data is more valuable for answering global concerns, while quantitative data validates accounts and answers fine-grained questions.

Bibtex entry

@INPROCEEDINGS { 1993-Woz.CHI,
CLASS = { CONFARTICLE },
AUTHOR = { Maulsby, D. and Greenberg, S. and Mander, R. },
TITLE = { Prototyping an Intelligent Agent through Wizard of Oz },
BOOKTITLE = { Proceedings of the ACM Conference on Human Factors in Computing Systems - ACM CHI'93 },
PAGES = { 277-284 },
YEAR = { 1993 },
MONTH = { May },
PUBLISHER = { ACM Press },
ADDRESS = { Amsterdam, The Netherlands },
NOTE = { Also as Report 1992-489-27, September },
}