TouchID Toolkit

General Information

The TouchID Toolkit enables developers to create applications on the Microsoft Surface that can identify different parts of the hand, hands and users. Additionally, custom postures and gestures can be integrated easily.

Basis is the Fiduciary Glove. It consists of an ordinary glove with fiduciary tags stuck onto 15 key hand parts. The Microsoft Surface is able to recognize these tags very reliably. The TouchID Toolkit offers high-level access to the hand parts, glove and users. Additionally, postures and gestures can be created, tested and integrated in applications.

Contents



Download and Installation

  • TouchID Toolkit (Release Version 1.5.0, July, 2011) Download of TouchID Toolkit installer (Sample applications included).
  1. Make sure that Visual Studio 2010 Express or Professional is installed
  2. Install the Surface SDK
  3. Download TouchID Toolkit installer and execute
  4. To create a TouchID Application, create a new project in Visual Studio 2010 and use the template provided.

Utility Manuals

Tutorials and Examples

  • Hand Part Tutorial. Learn how to differentiate between hand parts in your TouchID Application.
  • Gesture Tutorial. Learn how to integrate gestures in your TouchID Application.
  • Posture Tutorial. Learn how to integrate postures in your TouchID Application.

Code Snippets

For all snippets: 'this' is the TouchIDWindow of the TouchID Application.

Hands

  • Instantiating a Hand
Hand hand = new hand(this, "HandIdentifier");
  • Accessing hand parts
Handpart thumb = hand.Thumb;
Handpart indexFinger = hand.IndexFinger;
  • Subscribing to hand part events
// Event subscription
this.HandpartDown += new EventHandler<TouchEventArgs>(Window1_HandpartDown);
this.HandpartChanged += new EventHandler<TouchEventArgs>(Window1_HandpartChanged);
this.HandpartLeave += new EventHandler<TouchEventArgs>(Window1_HandpartLeave);

// Callback methods
void Window1_HandpartDown(object source, TouchEventArgs e){
    Console.WriteLine("Hand part down: "+e.Handpart.Identifier);
}

void Window1_HandpartChanged(object source, TouchEventArgs e){
    Console.WriteLine("Hand part changed: "+e.Handpart.Identifier);
}

void Window1_HandpartLeave(object source, TouchEventArgs e){
    Console.WriteLine("Hand part leave: "+e.Handpart.Identifier);
}
  • Subscribing to hand part type events
// Event subscription
this.FingerDown += new EventHandler<TouchEventArgs>(Window1_FingerDown);
this.KnuckleChanged += new EventHandler<TouchEventArgs>(Window1_KnuckleChanged);
this.PalmLeave += new EventHandler<TouchEventArgs>(Window1_PalmLeave);
this.BackOfHandEnter += new EventHandler<TouchEventArgs>(Window1_BackOfHandEnter);
this.SideOfHandUp += new EventHandler<TouchEventArgs>(Window1_SideOfHandUp);

// Callback methods
void Window1_FingerDown(object source, TouchEventArgs e){
    Console.WriteLine("Finger down: " + e.Handpart.Identifier);
}

void Window1_KnuckleChanged(object source, TouchEventArgs e){
    Console.WriteLine("Knuckle changed: " + e.Handpart.Identifier);
}

void Window1_PalmLeave(object source, TouchEventArgs e){
    Console.WriteLine("Palm leave: " + e.Handpart.Identifier);
}

void Window1_BackOfHandEnter(object source, TouchEventArgs e){
    Console.WriteLine("Back of hand enter: " + e.Handpart.Identifier);
}

void Window1_SideOfHandUp(object source, TouchEventArgs e){
    Console.WriteLine("Side of hand up: " + e.Handpart.Identifier);
}
  • Subscribing to hand part events with a particular hand
// Event subscription
hand.HandpartDown += new EventHandler<TouchEventArgs>(hand_HandpartDown);

// Callback method
void hand_HandpartDown(object source, TouchEventArgs e){
    Console.WriteLine("Hand part down"+e.Handpart.Identifier);
}
  • Subscribing to hand part events with a particular hand part
// Event subscription
hand.Thumb.HandpartDown += new EventHandler<TouchEventArgs>(thumb_HandpartDown);

// Callback methods
void thumb_HandpartDown(object source, TouchEventArgs e){
    Console.WriteLine("Thumb down");
}

Users

  • Instantiating a User
User user = new User(this, "UserName");
  • Assigning Hands to a User
Hand hand1 = new Hand(this, "HandIdentifier1");
Hand hand2 = new Hand(this, "HandIdentifier2");

// Alternative 1
user.LeftHand = hand1;
user.RightHand = hand2;

// Alternative 2
hand1.User = user;
hand2.User = user;
  • Subscribing to hand part events with a user. Events are received, if caused by hand parts on hands assigned to the user.
// Event subscription
Hand hand1 = new Hand(this, "HandIdentifier1");
Hand hand2 = new Hand(this, "HandIdentifier2");
hand1.User = user;
hand2.User = user;
user.HandpartDown += new EventHandler<TouchEventArgs>(user_HandpartDown);

// Callback methods
void user_HandpartDown(object source, TouchEventArgs e){
    Console.WriteLine("Hand part down: "+e.Handpart.Identifier);
}

Postures

  • Assigning a posture recognizer for a TouchID Application
this.PostureRecognizer = new PostureRecognizer();
  • Loading postures into the posture recognizer
// From the posture repository
this.PostureRecognizer.LoadPostures(PostureRepository);

// From another Uri
Uri folderWithPostures = new Uri(...);
this.PostureRecognizer.LoadPostures(folderWithPostures);
  • Subscribing to posture events
// Event subscription
this.PostureDown += new EventHandler<HandPostureEventArgs>(Window1_PostureDown);
this.PostureChanged += new EventHandler<HandPostureEventArgs>(Window1_PostureChanged);
this.PostureLeave += new EventHandler<HandPostureEventArgs>(Window1_PostureLeave);

// Callback methods
void Window1_PostureDown(object source, HandPostureEventArgs e){
    Console.WriteLine("Posture placed down: " + e.RecognizedPosture.Identifier);
}

void Window1_PostureChanged(object source, HandPostureEventArgs e){
    Console.WriteLine("Posture changed: " + e.RecognizedPosture.Identifier);
}

void Window1_PostureLeave(object source, HandPostureEventArgs e){
    Console.WriteLine("Posture leave: " + e.RecognizedPosture.Identifier);
}

Gestures

  • Assigning a gesture recognizer for a TouchID Application
this.GestureRecognizer = new GestureRecognizer(this);
  • Loading gestures into the posture recognizer
// From the gesture repository
this.GestureRecognizer.LoadGestures(GestureRepository);

// From another Uri
Uri folderWithGestures = new Uri(...);
this.GestureRecognizer.LoadGestures(folderWithGestures);
  • Subscribing to gesture event
// Event subscription
this.GestureRecognized += new EventHandler<HandGestureEventArgs>(Window1_GestureRecognized);

// Callback method
void Window1_GestureRecognized(object source, HandGestureEventArgs e){
    Console.WriteLine("Gesture: " + e.RecognizedGesture.Identifier + " performed by " + e.Handpart.Identifier);
}

GloveButton

  • Creating a GloveButton
// Button without text
GloveButton button = new GloveButton();

// Button with text
GloveButton button = new GloveButton("Press me.");
  • Setting the number of hand parts needed to press the button
button.HandpartsNeededToPressDown = 3;
  • Restricting button to hand part
// Restrict to hand parts with the specified identifier
button.RestrictToHandpart(Handpart.Identifiers.IndexFinger);

// Restrict to hand parts with the specified identifier and on the specified hand
button.RestrictToHandpart(Handpart.Identifiers.RingFinger, GloveModel.LeftOrRightHand.Right);

// Restrict to hand part with the identifier and on the hand specified by the HandpartModel.
HandpartModel leftThumb= new HandpartModel(Handpart.Identifiers.Thumb, GloveModel.LeftOrRightHand.Left);
button.RestrictToHandpartModel(leftThumb);
  • Button is pressable by an entering hand part (i.e. a hand part that already touches the surface and moves into the button area).
button.PressableByEnter = true;
>><<
//
  • Adjusting the appearance
// Size
button.Width = 100.0;
button.Height = 100.0;

// Unpressed appearance
button.Foreground = Brushes.Black
button.Fill = Brushes.LightGray;
button.Stroke = Brushes.DarkGray;
button.Effect = new System.Windows.Media.Effects.DropShadowEffect();

// Pressed appearance
button.ForegroundPressed = Brushes.White
button.FillPressed = Brushes.DarkRed;
button.StrokePressed = Brushes.LightRed;
button.EffectPressed = new System.Windows.Media.Effects.BlurEffect();;

Troubleshooting & Help

Links

Papers / Videos