Development Overview - jackspaceBerkeley/pupil GitHub Wiki

Overview of language, code structure, and general conventions

## Language Pupil is written in `Python`, but no "heavy lifting" is done in Python. High performance computer vision, media compression, display libraries, and custom functions are written in `C` and accessed through [ctypes][ctypes-pydoc]. Python plays the role of "glue" that sticks all the pieces together.

We also like writing code in Python because it's quick and easy to move from initial idea to working proof-of-concept. If proof-of-concept code is slow, optimisation and performance enhancement can happen in iterations of code. See the c_methods module, and the corresponding methods.c file for an example of how to do image filtering in C using Python, Numpy, and ctypes.

## Process Structure When [Pupil Capture][capture] starts, two processes are spawned: **Eye** and **World**. Both processes grab image frames from a video capture stream but they have very different tasks. ### Eye Process The eye process only has one purpose - to detect the pupil and broadcast its's position. The process breakdown looks like this: * Grabs eye camera images from eye camera video stream * Find the pupil position in the image * Broadcast/stream the detected pupil position.

Please note that, pupil position refers to the position of the pupil in the eye camera space. This is different from gaze position which is what we call the mapped pupil positions in the world camera space.

### World Process This is the workhorse. * Grabs the world camera images from the world camera video stream * Receives pupil positions from the eye process * Performs calibration mapping from pupil positions to gaze positions * Loads plugins - to detect markers, broadcast pupil positions over the network, and more... * Records video and data. Most, and preferably all coordination and control happens within the World process. ### Inter-Process-Communication IPC objects called `g_pool` connect the world and eye process. There are three IPC objects: ```python g_pool.pupil_queue = Queue() # a multiprocessing.queue g_pool.eye_rx, g_pool.eye_tx = Pipe(False) # a unidirectional multiprocessing.pipe g_pool.quit = RawValue(c_bool,0) # a shared boolean value ``` See where `g_pool` is created in the [source code][g-pool]. #### `pupil_queue` Python `Mulitprocessing.Queue` streams timestamped information from the eye process to the world process.

The pupil detector plugins, run by the Eye process are required to return a result in the form of a Python dictionary with at least the following content:

This dictionary is sent into the pupil_queue. The queue is emptied and read in the world process. Mapping from pupil position to gaze position happens here. The mapping function is supplied by a calibration plugin.

#### Control: World -> Eye One directional `multiprocessing.pipe` named with ends -- `g_pool.eye_rx` and `g_pool.eye_tx` -- is used to send simple control tokens to the eye process. Currently this is only used to start and stop recording of eye video. #### Quit! `g_pool.quit` is a `ctypes` boolean that tells the World process and Eye process to stop, clean up, and exit nicely. ## Timing & Data Conventions Pupil Capture is designed to work with multiple captures that free-run at different frame rates that may not be in sync. World and eye images are timestamped and any resulting artefacts (detected pupil, markers, etc) inherit the source timestamp. Any correlation of these data streams is the responsibility of the functional part that needs the data to be correlated (e.g. calibration, visualisation, analyses).

For example: The pupil capture data format records the world video frames with their respective timestamps. Independent of this, the recorder also saves the detected gaze and pupil positions at their frame rate and with their timestamps. For more detail see Data Format.

### Git Conventions We make changes almost daily and sometimes features will be temporarily broken in some development branches. However, we try to keep the `master` branch as stable as possible and use other branches for feature development and experiments. Here's a breakdown of conventions we try to follow. * `tags` - when we've finished making a new feature, or refactored code, we make a tag following the [semantic versioning][semver] protocol. Check out the [releases][releases]. * `master` - this branch tries to be as stable as possible - incremental and tested features will be merged into the master. Check out the [master branch][master-branch]. * `branches` - branches are named after features that are being developed. These branches are experimental and what could be called 'bleeding edge.' This means features in these branches may not be fully functional, broken, or really cool... You're certainly welcome to check them out and improve on the work!

If you've done something -- even if work-in-progress -- make pull request and write a short update to the Pupil community.