GSI Demo allows you to map existing single user applications to a multi user speech and gesture enabled tabletop by demonstration. Creating a mapping is as simple as saying "Computer, when I do [one finger gesture] you do [left mouse drag]" or "Computer, when I say layer bars you do [keyboard and mouse macro]". GSI Demo also provides facilities for rapidly prototyping true multi user speech and gesture applications from the ground up.
GSI Demo has several limitations: it does not support hand posture movements, it requires a Smart Board or a Diamond Touch table on a PC running Windows XP. For programmers, GSI Demo is implemented as a .NET package.
Please note that GSI Demo is a early prototype system that is provided 'as is'. While we would be happy to receive feedback, there are no resources to provide support and/or extensions to the existing system.
Tutorials and Examples
Using GSI Demo
- Getting Started. Learn how to Start a speech client and run an application mapping.
- Simple Mappings. Learn how to create your own speech and gesture mapping to Microsoft Windows Solitare.
- Advanced Mappings. Learn how to create a speech and gesture mapping to Mozilla Firefox.
Programming with GSI Demo
Notes about GSI Demo
Papers / Videos
- Tse, E., Shen, C., Greenberg, S. and Forlines, C. (2006) Enabling Interaction with Single User Applications through Speech and Gestures on a Multi-User Tabletop. Proceedings of Advanced Visual Interfaces (AVI'06), May 23-26, 336-343, Venezia, Italy, ACM Press.
- Tse, E., Greenberg, S., Shen, C. and Forlines, C. (2006) Multimodal Multiplayer Tabletop Gaming. Proceedings Third International Workshop on Pervasive Gaming Applications (PerGames'06), in conjunction with 4th Intl. Conference on Pervasive Computing, May 7th Dublin, Ireland. 139-148.
- Tse, E., Greenberg, S. and Shen, C. (2006) GSI DEMO: Multiuser Gesture / Speech Interaction over Digital Tables by Wrapping Single User Applications. Proc Eighth International Conference on Multimodal Interfaces (ICMI’06), (Nov 2-4, Banff, Canada), ACM Press.
- Tse, E., Greenberg, S. and Shen, C. (2006) Motivating Multimodal Interaction Around Digital Tabletops. Video Proceedings of ACM CSCW'06 Conference on Computer Supported Cooperative Work, November, ACM Press. Video and two-page summary. Duration 3:25.