Gesture Handling - KSU-CS-Software-Engineering/VetMedSim GitHub Wiki

This page describes the functionality of script(s) responsible for gesture recognition and their distribution to game objects. This approach allows us to handle user input on one global place, which saves both code space and computational time. The whole concept of distribution follows the Observer pattern.

Usage

First, you should create an empty game object, attach GestureHandler script to it and configure the recognition constants, if your project doesn't contain one already.

The recognition constants are the following:

  • ReactionTimeThreshold
    • the minimal time user has to hold finger on screen before Drag or Hold is recognized
    • the maximal time user can hold finger on display to be recognized as a Tap
    • recommended value: 1.0
  • TapDifferenceTimeThreshold
    • the maximal time that can pass in between multiple Taps in a Tap gesture
    • recommended value: 0.1 (unless you need multiple taps)
  • StationaryTimeThreshold
    • the maximal time that can Drag gesture remain stationary before it ends and new Hold gesture starts
    • recommended value: 0.4
  • StationaryDistanceThreshold
    • the maximal distance user can move their finger during Hold gesture before it ends and new Drag gesture starts
    • recommended value: 10

Afterward, let the control script of the game object, which should receive notifications about gestures, implement IGestureListener interface. This consist of two methods: OnGestureStart(Gesture) and OnGestureEnd(Gesture), which will execute once a gesture is recognized (resp. it ends).

You will also need to add public property referencing the global GestureHandler to the control script. When you have the reference available, you should call GestureHandler.RegisterListener( this ) as a part of the initialization. This method call will make the control script receive gesture notifications.

If you ever need for a game object to stop listening to gesture notifications, call GestureHandler.UnregisterListener( this ).

Every gesture has its type, which should be used for safe casting it to associated object. If you need to find more about the methods and properties of all gesture types, please see Gesture script for specific classes and their code documentation.

References scripts

  • UserInput/
    • Gesture.cs
    • IGestureListener.cs
    • GestureHandler.cs

Functionality

Currently, we can recognize four types of gestures:

  • One finger
    • Tap (single or multiple touch and fast release)
    • Hold (touch and release after a long while)
    • Drag (touch, move, and release)
  • Two fingers
    • Pinch (as drag but with two fingers)

If you need to recognize another gesture, please implement new class extending the abstract Gesture class and link it to the GestureHandler state machine. You can find more documentation and inspiration in code inside Gesture script.

GestureHandler processes the recognition of gestures using its state machine (SM). The SM starts in the Idle state. If new touch is recognized, it will move to the Gathering state.

The Gathering state will wait for the amount of time specified by the ReactionTimeThreshold property unless the touch registered in the Idle state ends. If that happens, it will automatically transition to the Tap state. After this amount of time passes, SM will identify the gesture and transition to the associated state by following rules:

  • if there are two or more touches on display -> Pinch state
  • if there is only one touch on the display
    • if the touch has ended -> Tap state
    • if the touch mas moved more than StationaryDistanceThreshold -> Drag state
    • else -> Hold state

At this moment SM will register the gesture by calling OnGestureStart(Gesture) on all registered listeners. SM will call the method OnGestureEnd(Gesture) after it finds out, that gesture ended (user lifted their finger off the display).

In the Tap state, SM will wait for the amount of time specified by the TapDifferenceTimeThreshold property. If the system registers an additional tap, the TapCount property will increase, and SM will repeat this step. Otherwise, the Tap gesture ends.

While in the Hold state, SM will keep track of how far user navigated their finger from the point of the first contact with the screen. If this distance exceeds the StationaryDistanceThreshold, SM will end the Hold gesture and start new Drag gesture with the same touch data.

If the user stops moving their finger while in the Drag state, SM will start a timer. If the value of this timer exceeds the StationaryTimeThreshold property, SM will end the Drag gesture and start new Hold gesture with the same data. If the user moves their finger before the value of timer reaches the StationaryTimeThreshold property, SM will reset the timer.

After any gesture terminates and is not replaced by another one (Drag and Hold state), SM will check, how many touches remain on the screen. If there is at least on touch, SM will transition to the Gathering state. Otherwise, it will fall back to the Idle state and wait for more touches.