Color Calibration

HSV Filtering

In order to track the a colored LED with a camera, we need to subtract the colored bulb from the rest of the background. This is done by:

1. Fetching the source RGB image from the camera
2. Converting the image into the Hue-Saturation-Value or "HSV" color space
3. Throw out all pixels that don't fall within the HSV ranges that correspond to tracking color

images/ColorFilter/HSVFilter.jpg

While every LED is set to a known tracking color, how each camera actually SEES the tracking color varies dramatically based on the lighting conditions. Therefore the HSV filter values can't just be set to some hard coded presets and be expected to work in all cases. We have to manually adjust the HSV filter to local lighting conditions.

images/ColorFilter/HSVRanges.jpg

Additionally, the exposure and gain camera settings need adjustment depending on lighting conditions to make the the LED pop in the video frame. Typically an exposure settings around 16-32 (on a PS3EYE) and a gain setting between 50-75 (on a PS3EYE) works in many lighting conditions.

images/ColorFilter/GainExposure.jpg

Future Work: Automatic Color Calibration

Ideally this color calibration would be done automatically. PSMoveAPI, the library that inspired this project, does an automatic blink calibration every time tracking is enabled. This process would try and compute an ideal exposure and saturation settings. Unfortunately it was difficult to get this to work in certain lighting conditions with some tracking colors and it was difficult to know how it was failing. So the approach I started with in this library is to set the HSV filter and camera settings manually and verify that they filter the tracking color correctly as a baseline.

There are some known techniques based on using image statistics to generate these thresholds more automatically. One approach I've seen mentioned that is supposed to better compute good HSV thresholds ia called "Histogram Backprojection". There is an example write up here on the technique