Pupil Player - jackspaceBerkeley/pupil GitHub Wiki

About

Pupil Player is the second tool you will use after Pupil Capture. It is a media and data visualizer at its core. You will use it to look at Pupil Capture recordings. Visualize your data and export it.

Features like marker tracking found in Pupil Capture are also available in Pupil Player, so that you can do after-the-fact surface tracking and analysis.

Starting Pupil Player

If you use the bundle (as know as "downloaded ready-to-go app" or "stand alone application") just drag the recording directory (the triple digit one) onto the Player icon.

Check out this video for a demonstration of how to start the Pupil Player bundle.

If you run from source do:

cd "path_to_pupil_dir/pupil_src/player"
python main.py "path/to/recording_directory"

Workflow

Pupil Player is similar to a video player. You can load plugins to build visualizations. Here is an example workflow:

• Start Pupil Player - drag the recording folder onto the app or follow instructions above if running from source
• Load plugins - for example Vis Circle
• See your visualization - press the play button to check out the visualization
• Set trim marks - you can drag the small circles on the ends of the timeline to set where the export will start and stop (you can also set trim marks in the Export Launcher plugin).
• Export - Load the Export Launcher plugin and click new export -- this will enable you to save out videos. It will run in the background and export your video with the current visualization settings. (Please make sure that your exports have a good filename.)

Note -- Pupil Player will never remove or overwrite any of your data gathered during capture.

Plugin Overview

Pupil Player uses the same Plugin framework found in Pupil Capture to add functionality. We implement all visualizations, marker tracking, and the exporter using this structure. Very little work (often no work) needs to be done to make a Capture Plugin work for the Pupil Player and vice versa.

There are two general types of plugins:

• Unique - You can only launch one instance of this plugin.
• Not unique - You can launch multiple instances of this type of plugin. For example, you can load one Vis Circle plugin to render the gaze position with a translucent green circle, and another Vis Circle plugin to render the gaze circle with a green stroke of 3 pixels thickness. You can think of these types of plugins as "additive."

In the following sections we provide a summary of plugins currently available and in Pupil Player.

Visualization Plugins and Utilities

For the sake of clarity, we will call plugins with the Vis prefix visualization plugins. These plugins are simple plugins, are mostly additive ( or not unique), and directly operate on the gaze positions to produce visualizations. Other plugins like Offline Marker Detector also produces visualizations, but will be discussed elsewhere due to the extent of its features.

Vis Circle

Visualize the gaze positions with a circle for each gaze position. This plugin is not unique, therefore you can add multiple instances of the plugin to build your visualization. You can set the following parameters:

• radius - the radius of the circle around the gaze point.
• stroke width - the thickness or width of the stoke in pixels.
• fill - toggle on for a circle with solid fill. Toggle off for a circle with only stroke.
• color - define the red, green, blue values for color. Alpha defines the opacity of the stroke and fill.

This image is an example of how you could use two instances of the Vis Circle Plugin. The first instance renders the gaze position as an opaque green circle with a radius of 40 pixels. The second instance renders the gaze position as a stroked red circle, with stroke width of 5 pixels and a radius of 40 pixels.

Vis Cross

Visualize the gaze positions with a cross for each gaze position. This plugin is not unique, therefore you can add multiple instances of the plugin to build your visualization. You can set the following parameters:

• inner offset length - the distance in pixels to offset the interior cross endpoints from the gaze position. A value of 0 will make the crosshairs intersect the gaze position.
• outer length - The length of the cross lines in pixels from the gaze position. Note - equal values of inner offset length and outer length will result in a cross with no length, and therefore not rendered.
• stroke width - the thickness or width of the stoke in pixels.
• color - define the red, green, blue values for color.

This image is an example of how you could use two instances of the Vis Cross Plugin. The first instance renders the gaze position as a red cross with that extends to the boundaries of the screen. The second instance renders the gaze position as a green cross, with a heavier stroke weight.

Scan Path

This plugin enables past gaze positions to stay visible for the duration of time specified by the user. This plugin is unique, therefore you can only load one instance of this plugin.

On its own, Scan Path does not render anything to the screen. It is designed to be used with other plugins. In some cases, it is even required to be enabled in order for other plugins to properly function. When used with Vis plugins (like Vis Circle, Vis Cross, Vis Polyline, or Vis Light Points) Scan Path will enable you to see both the current gaze positions and the past gaze positions for the specified duration of time.

An example showing Scan Path of 1 second duration with Vis Circle. Each green circle is a gaze position within the last 1 second of the recording.

Vis Polyline

Visualize the gaze positions with a polyline for each gaze position. This plugin is not unique, therefore you can add multiple instances of the plugin to build your visualization. You can set the following parameters:

• line thickness - the thickness or width of the polyline stroke in pixels.
• color - define the red, green, blue values for color.

An example showing Vis Polyline used with Vis Circle and Scan Path. The polyline enables one to understand the sequence of the gaze positions over the duration specified in Scan Path.

Vis Light Points

Visualize the gaze positions as a point of light for each gaze position. The falloff of the light from the gaze position is specified by the user. This plugin is not unique, therefore you can add multiple instances of the plugin to build your visualization. You can set the following parameters:

• falloff - The distance (in pixels) at which the light begins to fall off (fade to black). A very low number will result in a very dark visualization with tiny white light points. A very large number will result in a visualization of the world view with little or no emphasis of the gaze positions.

An example showing Vis Light Points with a falloff of 64.

Filter Fixations

A simple filter that removes gaze positions that are greater than a specified distance away from the the previous gaze position. This plugin is unique, therefore you can only load one instance of this plugin. Filter Fixations depends on Scan Path. You will see a warning if Scan Path is not loaded.

Manual Gaze Correction

This plugin allows one to manually offset the gaze position. The offset values are between -1 and 1. This plugin is unique, therefore you can only load one instance of this plugin. You can set the following parameters:

• x_offset - the amount to offset the gaze position horizontally
• y_offset - the amount to offset the gaze position vertically

Eye Video Overlay

This plugin can be used to overlay the eye video on top of the world video. Note that the eye video is not recorded by default in Pupil Capture, so if you want to use this plugin, make sure to check record eye video in Pupil Capture. This plugin is unique, therefore you can only load one instance of this plugin. You can set the following parameters:

• opacity - the opacity of the overlay eye video image. 1.0 is opaque and 0.0 is transparent.
• mirror image - you can mirror (flip) the image over the Y axis.

An example of the Eye Video Overlay plugin used in conjunction with visualization plugins Scan Path, Vis Circle, and Vis Polyline.

Offline Marker Detector

This plugin is an offline version of the Marker Tracking plugin for Pupil Capture. You can use this plugin to detect markers in the recording, define surfaces, edit surfaces, and create and export visualizations of gaze data within the defined surfaces.

Click the image above to link to a demo video for Pupil Player Offline Marker Detector.

Here is an example workflow for using the Offline Marker Detector plugin to generate heatmap visualizations and export surface data reports:

• Load Offline Marker Detector plugin - if you already have surfaces defined, the load will take a few seconds because the plugin looks through the entire video and caches the detected surfaces.
• Add surface - if you do not have any defined surfaces, you can click on the Add surface button when the markers you want to user are visible or just click the circular A button in the left hand side of the screen.
• Surface name and size - In the Marker Detector GUI window, define the surface name and real world size. Note - defining size is important as it will affect how heatmaps are rendered.
• Set trim marks - optional, but if you want to export data for a specific range, then you should set the trim marks.
• Recalculate gaze distributions - click the (Re)calculate gaze distributions button after specifying surface sizes. You should now see heatmaps in the Player window (if gaze positions were within your defined surfaces).
• Export gaze and surface data - click the Export gaze and surface data button to export surface metric reports. Reports will be saved into your recordings folder.

Export Video with Visualizations

You can export video with visualizations with the Export Launcher plugin or apply visualizations to a whole directory (folder) of recordings using the Batch Exporter. By default the first visualization will be named world_viz. Note - Pupil Player will not overwrite your visualizations.

Export Launcher

To export a video, load the Export Launcher plugin. You can select the segment of the video you want to export by setting trim marks in the timeline slider or directly in the plugin GUI. You can specify the name of the export in the GUI. Click new export to start the export. The exporter will run in the background and you can see the progress bar of the export in the GUI. While exporting you can continue working with the video.

Batch Exporter

You can use this plugin to apply visualizations to an entire directory (folder) of recordings in one batch. You need to specify the following:

• Recording source directory - a directory (folder) that contains one or more Pupil recording folder.
• Recording destination directory - an existing directory (folder) where you want to save the visualizations.

Developing your own Plugin

To develop your own plugin see the developer guide.