Currently used solutions - HelmchenLabSoftware/contrast-marker-video-tracker GitHub Wiki

Currently used approach is a combination of automatic and manual processing, where the user does not have to explicitly specify regions of interest, but has to manually tweak the algorithm parameters to eliminate spurious regions and help algorithm navigate during the situations where the correct regions are not visible. The approach can be subdivided into the following stages

1. Preprocessing

Thanks to the enhanced visibility of retro-reflective beads, the ROI mostly stand out in terms of contrast. Good results are achieved by

Applying a background subtraction routine
Select a narrow contrast range, which is specific to ROI.

Note that this operation needs to be performed only once. This is a very fast procedure, but it relies on the assumption that the background contrast does not change during the trial.

2. ROI automatic identification

A simple Breath-First-Search algorithm, (BFS, known as as bwconncomp in Matlab) can be used to partition a frame into nonzero connected components – the ROI. Clearly, the naive algorithm generates too many regions. Some fake ROI appear due to noise. Some real regions can split because of temporary loss of reflection in the middle of the region under certain angles, or because of another object getting in the way. The most straightforward approach to address this problem is to

Clustering: Merging all
Deleting all regions that are too small, thus reducing noise.

The major advantage of this approach is that it only depends on the current frame, and is therefore fault tolerant to temporary disappearance of regions.

The major disadvantage is that the experiment is not consistent with the assumptions behind this strategy. For example, marks on the animal fingers may appear and disappear, cross, and get close enough to effectively merge. About half of the time, less than 3 ROI are visible. Due to this, precise identification of traces of all 3 ROI on the fingers seems infeasible. Instead, one can compute the average trace of all 3, as well as estimate "collectedness" of the paw by counting the number of visible regions within the area.

3. Time-connecting ROI into traces

A recombination algorithm is necessary to combine separated regions together based on the shortest distance. A further algorithm must connect regions disjoint in time according to overlap in their trajectories.

Initially, it is not obvious how are regions connected from frame to frame. At first, connecting regions according to shortest distance makes sense. If there are extra regions unrelated to markings, they will appear and disappear, and may interfere with the above procedure The extra regions will need to be deleted afterwards. Firstly, delete all regions that are too big or too small to be relevant. Then automatically deleting all regions that only persist in small part of the movie, and then manually deleting entire sequences of persistent regions that are not related to markings.

4. Time-connecting traces

5. Filtering

Finally, all remaining traces that connect to nothing or are persistent noise can be filtered by Deleting all ROI within certain coordinate and size range Deleting all traces that are too short and have failed to connect to anything

After filtering, the time-connecting traces can be repeated to improve results.