_{^{These instructions are for VSDK Unity. For VSDK Unreal documentation, see the VSDK Unreal wiki.}}

This section assumes you have finished Chapter 1.

This section is more advanced, but introduces hand-based interactions to show off the power of VSDK. Interacting with virtual environments using your own hands is incredibly immersive. By using hand tracking technology, we can put the user’s own hands directly into VR. For this section, we will be using the Leap Motion hand tracking system as it is widely available and delivers high-quality results for the relatively inexpensive cost.

Additionally, note that Leap Motion is only available on Desktop systems at this time and will not be compatible with any standalone XR systems.

First, we will set up the VSDK Leap Motion SDK.

• Find the SDK Setup object for the XR SDK you are using and in the ‘Hands’ drop down select Leap Motion.

Next, prepare the Leap Motion rig:

• Find the Hands object under [VRTK_SDKManager]. If it has any children, delete them.
• Navigate to Assets/LeapMotion/Core/Prefabs and drag Leap Rig into the hierarchy as a child of Hands. Rename it ‘Leap Motion’
• Click on the ‘Main Camera’ object under Leap Rig and uncheck Audio Listener and Camera. These components are necessary for Leap XR Service Provider but unfortunately add an additional camera to the scene that interferes with the VSDK camera.
• On the Leap XR Service Provider component, set Physics Extrapolation to None and Temporal Warping Mode to Off
• Click on the Hand Models object and expand the model pool. If Capsule Hands is in the pool, uncheck the Is Enabled field.
• Right click on Hand Models and add an empty child object named ‘Tutorial Hands’
• Navigate to Assets/LeapMotion/Core/Prefabs/HandModelsNonHuman. Drag LoPoly Rigged Hand Left and LoPoly Rigged Hand Right into the hierarchy as children of Tutorial Hands
• Click on Hand Models again and increase the size of the Model Pool by one. Set the new group name to Tutorial Hands and drag the Drag LoPoly Rigged Hand Left object into the Left Model and the Drag LoPoly Rigged Hand Right into the Right Model. Check Is Enabled.

Now, you should be able to see your hands in XR! However, you won’t be able to interact. Let us fix that.

• Add an empty child to Leap Motion and name it ‘Gesture Library’
• Add the SDK_LeapMotionGestureLibrary component to the Gesture Library object
• Navigate to Assets/LeapMotion/Core/Prefabs and drag the Attachment hands prefab into the scene as a child of Hand Models
• The Attachment Hands prefab provides objects synced to particular locations on the leap hands that you can use to precisely place objects. For this example, we are only using the palm.
• Right click on the Attachment Hands and select ‘Unpack Prefab’
• Under the Attachment Hand (Left) object, add an empty child to the palm object and name it ‘Hand Setup (Left)’
  • Add a sphere collider, check is trigger, and scale it to 0.1 in each axis
  • Next, add the Gesture Controller Event component to the palm object. Verify that the Controller Hand Id is set to Left.
  • Click the Grip Gesture field and select the Grab gesture
  • Click the Trigger Gesture field and select the Pinch gesture
  • Add the VRTK_InteractTouch component to the palm object
  • Add the VRTK_InteractGrab component to the palm object. Drag the Gesture Controller Event component into the Controller Events field and the VRTK_InteractTouch component in the VRTK_InteractTouch field
  • Add the VRTK_InteractUse component to the palm object. Drag the Gesture Controller Event component into the Controller Events field ,the VRTK_InteractTouch component in the Interact Touch field, and the VRTK_InteractGrab component into the Interact Grab field.
  • Add a Rigidbody component and check Is Kinematic and uncheck Use Gravity
• Duplicate the Hand Setup (Left) object with CTRL-D and move the copy to be a child of the Attachment Hand (Right)/Palm object
  • Change the Gesture Controller Event component Controller Hand Id field to Right
  • Reset the transform Position and Rotation values to 0.
• Finally, disable the top level Leap Motion object so that it is turned off by default and only activated if it is enabled by an SDK setup

At this point, you should be able to grab objects using your leap hands! All you have to do is close your first naturally. The grab points will be off, so you may need to adjust them manually as discussed for the welder to make the grab feel more natural. Some users are tempted to use the pinch gesture popularized by the Hololens to grab objects; this gesture is unnatural for anything but small objects. We recommend using a naturalistic grab instead (i.e., where a user grabs by closing their fist).

The hands are also set up to activate a trigger using the Pinch gesture.

At this point, the grab points on the objects are going to feel unnatural. When working with tracked hands, taking the time to set up GrabAttach scripts on objects that use both a left and right hand grab point is essential. See the screenshots below for good settings for the Welder to get you started.