BMVC 2016

[bmvc16_142] Beyond Action Recognition: Action Completion in RGB-D Data [PdF] [notes]

Farnoosh Heidarivincheh, Majid Mirmehdi, Dima Damen

read 2019/07/08

Objective

Introduce the notion of action completion, to distinguish success in actions.

Challenge the fact that one-vs-all action classification is not the most relevant framing for some end tasks

Analyze whether recognizing action completion requires specific features (compared to action recognition).

Inroduce a dataset for action completion: RGBD-Action-Completion-2016

Motivation

Detect action completion for instance to detect if elderly persons have completed tasks such as taking medicine or closing the tap

Synthesis

Focus on object interactions.

Feature candidates

• skeleton based features ++- joint positions, relative to a root joint ++- joint relative positions (difference between every pair of joints in the same frame) ++- joint velocities

Method

Frame completion as a binary classification task. Cross-validate by keeping one subject out

Dataset

8 subjects, performing 6 actions:

switch: subjects were asked to pretend they have forgotten to switch the light off, plug: subjects were given a plug that does not match the socket, open: a lid was glued to the jar so it could not be opened, pull: a drawer was locked so could not be pulled, pick: subjects were asked to attempt to pick an object, and then change their mind, drink: a mug was filled with very hot water unsuitable for drinking.

Experiments

Compare the different features for different tasks (action classification, action completion)

Notes

Next litterature

X. Wang, A. Farhadi, and A. Gupta. Actions ~ transformations. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016.