SIGMA Data Streams

The SIGMA system can collect a variety of sensor data streams from the HoloLens 2 device. The basic sensor streams collected in the default configuration are listed below:

Stream	Representation	Frame rate
RGB Camera	896 x 504, NV12 encoding	5 Hz
Preview Camera	896 x 504, NV12 encoding	5 Hz
Depth Camera	320 x 288, 16bpp	5 Hz
Eye Gaze	position (3 x 1 vector), direction (3 x 1 vector)	~30 Hz
Head	pose (4 x 4 matrix)	~30 Hz
Hands	poses (4 x 4 matrix) for each of the 26 joints in the left and right hand	~20 Hz
User Audio	1-channel, 16-bit PCM	16 kHz
System Audio	1-channel, 16-bit PCM	16 kHz

In addition to images, all camera streams (RGB, preview, and depth) also contain the corresponding intrinsics and extrinsics parameters.

The system logs a variety of other streams computed at runtime, such as speech recognition results, user interface commands, etc. The full set of captured streams can be inspected by opening one of the collected stores in Platform for Situated Intelligence Studio.