.
.
.
.
.
.
.
.

WSRKinectSensor store color, depth and skeleton frame to be processed by dedicated Tasks.

The parameter only will only start speech recognition and stop other features. The parameter fps will decrease frame per second and soft CPU usage.

custom.ini

; speech only do not start other features (for low cpu)
only=false

; Kinect global FPS (1 = 30fps; 2 = 15fps; 3 = 10fps; 5 = 6fps)
fps=2

; Sensor elevation +/- 27
elevation=0

• Use: Depth Frame
• Module: Internal

Trigger a StandBy mode after a given amount of time.

• The StandBy mode relies on Depth data.
• When activated other Tasks are suspended.

custom.ini

; recognize motion (default is 200ms)
motion=200

; threashold % to detect motion (default 7%)
motionTH=7

; timeout starting stand-by (default 5*60*1000 = 300000 = 2 minutes)
standby=300000

• Use: Color Frame
• Module: HUE

Compute most prominent color of a color frame. Send the color to server every N milliseconds. Can be use to trigger HUE Plugin.

custom.ini

; detect most prominent (default is 0ms)
color=45

; time in millisec between 2 prominent color
colorTH=0

• Use: Color Frame
• Module: HTTP
• Library: ZXing.Net

Seek for QRCode in color frame every N milliseconds. Send URI using HTTP Request or vocalize text.

• Skeleton tracking prevent QRCode (user is far from camera)
• Do not works with other BarCodes because of low resolution camera

custom.ini

; recognize qrcode (default is 200ms)
qrcode=200

; time in millisec before next QRCode (default is 2000ms)
qrcodeTH=2000

• Use: Skeleton Frame
• Module: Gesture

Check all gestures described in plugins/*.gesture then fire HTTP Request. The recognition is skipped if the skeleton is messed up.

• Keep track of skeleton height
• Keep track of head location

custom.ini

; recognize gesture (default is 45ms)
gesture=45

; time in millisec before next gestures (default is 1000ms)
gestureTH=1000

; distance between head and foot must be more than this size in cm to avoid bug (defaut 80cm)
gestureFix=80

; Use seated gesture
seated=false

; Start gesture in StandBy mode (waiting for voice command)
gestureSB=false

Gesture are described in an XML file named {plugin}.gesture. Gesture recognition is performed by checking 3D position of 2 joints with each other.

<gesture description="Hands Up" maxExecutionTime="1500" url="http://127.0.0.1:8080/sarah/gesture?g=5">
  <component firstJoint="WristLeft"      beginningRelationship="BelowAndLeft"  
             secondJoint="ShoulderLeft"  endingRelationship="AboveAndLeft" />
  <component firstJoint="WristRight"     beginningRelationship="BelowAndRight" 
             secondJoint="ShoulderRight" endingRelationship="AboveAndRight" />
</gesture>

Limitations:

• Do not overlap gesture
• Multiple component in a gesture is complicated to perform
• Use gesture plugin to trigger rules if possible

Troubleshooting:

Debug is complicated, use logs to see when a request is sent. Sometimes Kinect is too smart and messed up skeleton. The gestureFix check if foot and head are too close.

Height

The gesture manager try to guess user height based on it's forearm.

Head position

The gesture manager store latest X,Y,Z head location.

See also:

• SARAH: Reconnaissance gestuelle

• Use: Color Frame
• Module: Face
• Library: Emgu CV

This feature relies on 3 independant tasks to optimize CPU. You should remove your glasses during trainning to improve recognition

Use a HaarCascade algorithm to detect faces rectangle.

Use EigenObjectRecognizer to compare matching faces with known database. The recognition task keep track of seen faces in the ProfileManager.

Save an image of a new trained face in profile/faces/*. The image name contains user's id. A user can have multiple trained faces.

Track 87 head point and head animation to guess user Mood.

; detect faces position (default is 45ms)
facedetec=45

; recognize faces (default is 200ms)
facereco=200

; track faces 3D Shapes (default is 45ms)
facetrack=45

; timeout in millisec for a given face (5*60*1000)
faceTH=300000

; Start face in StandBy mode (waiting for voice command)
faceSB=false