Exploring True Multi-User Multimodal Interaction over a Digital Table

Tse, E., Greenberg, S., Shen, C., Forlines, C. and Kodama, R. (2008)
Exploring True Multi-User Multimodal Interaction over a Digital Table. In Proceedings of the ACM Conference on Designing Interactive Systems - ACM DIS'08. (Cape Town, South Africa), ACM Press, pages 109-118, February 25-27. See related video report iLab-2008-01. Earlier version as Report 2007-877-29, August.

View Publication and Related Materials

PDF PaperPDF Paper (2008-TrueMultimodal.DIS.pdf)

Abstract

True multi-user, multimodal interaction over a digital table lets co-located people simultaneously gesture and speak commands to control an application. We explore this design space through a case study, where we implemented an application that supports the KJ creativity method as used by industrial designers. Four key design issues emerged that have a significant impact on how people would use such a multi-user multimodal system. First, parallel work is affected by the design of multimodal commands. Second, individual mode switches can be confusing to collaborators, especially if speech commands are used. Third, establishing personal and group territories can hinder particular tasks that require artefact neutrality. Finally, timing needs to be considered when designing joint multimodal commands. We also describe our model view controller architecture for true multi-user multimodal interaction

Keywords

Tabletop interaction, multimodal speech and gesture interfaces, computer supported cooperative work, ACM: H5.2 [Information interfaces and presentation]: User Interfaces. – Interaction Styles

Bibtex entry

@INPROCEEDINGS { 2008-TrueMultimodal.DIS,
CLASS = { CONFARTICLE },
AUTHOR = { Tse, E. and Greenberg, S. and Shen, C. and Forlines, C. and Kodama, R. },
TITLE = { Exploring True Multi-User Multimodal Interaction over a Digital Table },
BOOKTITLE = { Proceedings of the ACM Conference on Designing Interactive Systems - ACM DIS'08 },
YEAR = { 2008 },
MONTH = { February 25-27 },
ADDRESS = { Cape Town, South Africa },
PUBLISHER = { ACM Press },
PAGES = { 109-118 },
NOTE = { See related video report iLab-2008-01. Earlier version as Report 2007-877-29, August },
KEYWORDS = { Tabletop interaction, multimodal speech and gesture interfaces, computer supported cooperative work, ACM: H5.2 [Information interfaces and presentation]: User Interfaces. – Interaction Styles },
}