CVPR 2013

Modeling Actions through State Changes[PdF] [notes]

Alireza Fathi and James M. Rehg

read 2019/08/05

Objective

Learn the object states that allow to recognize daily actions. Use these object state changes to recognize the action. Focus on first person cooking scenarios. Show improvement on action recognition and action segmentation.

Method

Focus on:

• state changes (closed, open)
• transformation of stuff (spread, mixed, ...)

Identify changed regions

Change detection

• Find regions that appear or disappear throughout action instances
• sample several start and end frames form the instance
• match pixels using large displacement optical flow
• compute change based on color difference
• compute significance of change based on average amount of change in pixels
• between beginning frame and target frame keep the one with minimum change (min color changes accross warped pixels)

Find consistent regions

• similar to k-means but with constraints that enforce a cluster to have samples from at least a certain number of action instances

State-specific detector

• train a linear SVM by using regions belonging to the cluster and regions
• each region is described with color (128-dim color histogram) texture and shape (similar to HOG features)

Action recognition

• given test action interval, get first and last frame, and concatenate outputs from state-specific detector
• linear SVM to train classifier for each action type

Action segmentation

• for each action class train 2 state detectors, one on beginning frames, one on ending frames, on top of the frame's response to pre-trained state-specific region detectors