Hand Pose Estimation

Advantages

First person view

• Good viewing perspective to analyze hand-object interactions
• Possibility of continuous recording of natural hand interactions

Challenges

• Uniform appearance

  • uniform color
  • redundant patterns

• Occlusion

  • self occlusion
  • occluded by object during object manipulation

• Ground Truth acquisition

  • 3D estimation is challenging

Main approaches

• Discriminative : train a classifier to learn a mapping from observations to poses

  • used to estimate hand pose from a single frame
  • Advantages
    • Faster
    • do not require initialization
  • Challenges
    • less accurate

• Generative : optimization problem

  • objective function that quantifies the discrepancy between visual objervations from 3D image senfor and 3D hand model hypothesis
  • Advantages
    • exploit time continuity
    • more accurate
  • Challenges
    • often non-differentiable and with local minima
    • computationally expensive

• Hybrid

  • provide a first estimation by classifier (discriminative)
  • optimize this solution using a model (generative)

Datasets

• can be acquired using equipment that modify the appearance of the hand (can still be used for depth maps)

  • magnetic sensor
  • inertial sensors