Head mounted eye tracking with 3D lenses - nimh-nif/SCNI_Toolbar GitHub Wiki
To take advantage of the SCNI's immersive virtual-reality dome projection environment, we wanted to record both neural signals and eye position (in screen-centered coordinates) in head-free animals. This poses several technical challenges that we are currently working to address:
- The head-mounted video eye-tracking apparatus must be sufficiently light and robust for extended use by chaired animals.
- The frames that hold the polarizing filters of the 3D glasses and the eye cameras in front of the eyes must minimize the portion of the visual field that they occluded.
- Head direction signals (in world-centered coordinates) must be integrated with eye direction signals (in head-centered coordinates) in order to compute gaze direction in world-centered coordinates online.
1) PupilLabs: A light-weight head-mounted eye tracker
The SCNI typically uses 'Primate mount' EyeLink II cameras from SR Research in head-fixed neurophysiology experiments. However, while these cameras were intended to be potentially worn head-mounted on human subjects, they are relatively large and heavy, thus obstructing the visual field and putting strain on the wearer. We therefore opted to use a new open-source system from Pupil Labs. Many of the camera specs are comparable to those of the EyeLink II (see below and here for a more in depth comparison of performance), but the hardware are substantially smaller and lighter. This allows them to be discretely mounted below the eye with a direct line of sight - thus avoiding the need for cumbersome and delicate hot mirrors, while still minimizing occlusion of the subject's visual field.
| SR Research EyeLink II | Pupil Labs | |
|---|---|---|
| Sampling frequency | 500 Hz | 200 Hz |
| Gaze accuracy | 0.5 deg | 0.6 deg |
| Gaze precision | 0.01 deg | 0.08 deg |
| Camera Latency | 3 ms | 4.5 ms |
| Weight | XXX g | 32 g |
| Cost | $20,000 | $2,000 |
The eye cameras are very small and can be easily removed from the black 3D-printed casing provided by PupilLabs for human head-mounted use. Instead, we designed a lower-profile camera casing to integrate with the 3D glasses frames that the macaque subjects wear, as seen in the photos below.
To maximize the subject's field of view in the dome environment, we designed a pair of glasses frames to closely fit the contours of the face of an average macaque (see computer renders below). The wrap-around frame style helps to bend the circular polarizing filters (taken from commercial 3D glasses) at the temporal edge, thus maximizing field of view and preserving continuity across the visual field.
To allow adjustment of the glasses position for animals with different face shapes and yet keep the glasses firmly in place during head movement, the 3D-printed frames attach to an adjustable mounting system that is anchored to the subject's headpost.