Basic Workflow - jackspaceBerkeley/pupil GitHub Wiki

An example workflow for eye tracking with Pupil can be summarized below. This page serves as quick notes on a workflow with Pupil, much more detailed information in the user guide is linked from each one of the overviews. Click topics for more detail.

• Plug in cameras
• Put on the Pupil headset
• Start Pupil Capture software
• Focus cameras
• Check pupil detection
• Calibrate
• Record
• Visualize

## Start Pupil Capture If you downloaded and installed the Pupil Capture bundle, you will see a Pupil icon. Just double click the Pupil icon and and you should see 2 windows – `Eye` and `World`. If cameras are *not* plugged in, both windows will still load but with colored squares for World and gray squares for Eye. Skim through the notes on Eye and World windows below.

• Eye - displays a video feed of your eye, camera controls for the eye camera, and controls and visualization for eye tracking algorithms.
• World - displays a video feed of your field of view (FOV), camera controls for the world, controls for plugins like calibration, recording, streaming, and more.

Still have questions? Read more about the Pupil Capture environment.

## Focus Cameras Ok, this may seem obvious, but you need to focus your the cameras in order to get good eye tracking performance and a good video from the World camera. Good news is that focal settings are saved, so you only have to focus once unless you unplug the Pupil headset. Here is an overview of the process. * Eye * Disable auto focus (Dev only) * Manually focus in Eye window GUI (Pupil Dev). Focus the camera by adjusting the arm or rotating the lens (Pupil Pro) * If details of the iris (the color part of your eye) are seen clearly, then you're done. * World * Set the focus for the distance at which you will be calibrating. * You can toggle the auto focus button on/off until the camera focuses itself at your desired depth or focus manually. * We recommend disabling auto focus for calibration. You can always turn auto focus back on after calibration. ## Check Pupil Detection After focusing, take a look at the Eye window. If the pupil is detected you will see a red circle around the edge of your pupil and a red dot at the center of your pupil. Try moving your head around a bit while looking at your eye to see that the pupil is robustly detected. In most cases, it should just work without having to fiddle with any of the controls. But, let's say you're moving around a really bright environment and your pupil is very small or constricted. Then you might want to change the minimum and maximum threshold for better performance, and maybe even limit the region of interest (ROI).

See this video for a demonstration of how to set the ROI and pupil min/max diameter.

• Region of Interest (ROI)
  • Open the main sidebar GUI by clicking on the toggle (upper right corner).
  • In the General menu switch to the Mode titled ROI.
  • A white rectangle and blue circle handles will appear.
  • Drag the handles to change the size of the ROI rectangle. Everything outside the rectangle will be ignored by the pupil detection algorithm.

ROI edit mode with sidebar menu collapsed. ROI is the area within the white rectangle.

• pupil min/max diameter
  • In the 'General' menu in the sidebar, switch the Mode to 'Algorithm`
  • Scroll down to the Pupil Detector submenu
  • Adjust Pupil min and Pupil max sliders.
  • Red circles in the bottom left visualize Pupil min and Pupil max values.
  • The green circle and value displayed at the center of the circles is the current pupil diameter. Make sure the green circle is within the two red circles.

Algorithm view mode, with sidebar menu open. Pupil min/max diameter sliders are in the sidebar menu and visualization of diameters in bottom left of the window.

See Pupil Capture for more details about pupil detector settings and advanced options.

## Calibrate Pupil uses two cameras. One camera that captures a subject's eye movements - Eye Camera - and another that records the subject's FOV - World Camera. In order to know what someone is looking at, we first need to find out how these two streams of information are correlated. This is what we call calibration. We will outline a basic workflow using the `Screen Marker` calibration method.

Check out a video walkthrough using the Screen Marker calibration method here.

• Double check that your pupil is robustly detected in the Eye window.
• Open the main sidebar GUI by clicking on the toggle (upper right corner).
• Double check that the World camera is focused for the screen and is not over-exposed (you can change exposure settings) in the Camera settings sub-menu.
• In the Calibration sub-menu select Screen Marker Calibration as the method.
• In the Calibration>Controls sub-menu:
  • Choose the monitor from the drop down menu (if you have more than one monitor).
  • Click fullscreen (recommended) to display the calibration animation in full screen mode.
• Press the c key on your keyboard or press the circular C button in the left hand side of the world window.
• Follow the marker on the screen with your eyes and try to keep your head stationary.
• After calibration is complete, the calibration window will close
• In the General menu Open Plugin>Show Calibration to evaluate the calibration quality. Quick note -- in a good calibration: the Number of used samples should be more that 180 and the Fraction of used data points should be greater than 0.75.

There are other calibration methods and lots more information about how calibration works -- read more on calibration.

## Record Once calibrated, you can start capturing data. This means that the capture software will save the world video stream and all corresponding gaze data in a folder. Quick steps: * Start recording: Press the `r` key on your keyboard or press the circular 'R' button in the left hand side of the world window. * The elapsed recording time will appear next to the 'R' button. * Stop recording: Press the 'r' key on your keyboard or press the circular 'R' button in the left hand side of the world window.

See a video demonstration of how to set recordings path, session name, and start recording -- here.

By default, each recording will live in its own unique data folder contained in the pupil_recordings folder. You can make as many recordings as you like. No recording will be overwritten. An example default folder structure that contains three recording sessions would look like this.

• pupil_recordings
  • 2015-02-24
    • 001
    • 002
    • 003

The location or Path to recordings and Session name can be changed within the Recorder sub-menu in the GUI. Note -- you must supply an existing and valid path for your Path to recordings.

You can turn on the Record eye switch in the Recorder sub-menu. The eye video will be saved in the same directory as the world video and all other data.

Read more about recording.

## Visualize Now that you have recorded some data, you can play back the video and visualize gaze data, marker data, and more.

For a detailed explanation of functionality see:

• Pupil Player - playback and visualization software.