Refining parameters - ju1ce/April-Tag-VR-FullBody-Tracker GitHub Wiki

This page will describe the main parameters that you will want to finetune in order to get the best experience you can. Since this may vary based on your hardware and your needs, feel free to experiment with them! Do modify one parameter at a time, so if something goes wrong, you know what parameter is to blame.

Camera settings

I have talked about this in the camera section, but having your camera settings correct is crucial to get good detection. Make sure your camera is opened in 720p or 1080p and at full refresh rate. Make sure your exposure and gain are set properly as well! You want exposure to be low enough to prevent motion blur, but going overboard may make the image too dark for detection to properly work. Refer to the Connecting a camera section!

Quad decimate

Quad decimate defines how much is the image downscaled before initial marker detection. This means that, having a higher number, detection will run much faster, but the maximum range is reduced. VR playspaces are relatively small, so this parameter helps speed things up at virtualy no cost.

To find out the optimal value, you will have to test them. When in VR and your trackers are tracked, there will be a gray number in the upper left corner, which determines the time ATT needed to process last frame. With a 30fps camera, you want this to be under 0.033 to ensure that no frames are dropped. If the number is higher than that, you want to increase quad decimate by 1, save, and restart detection (press 6.Start/stop first to stop, then to start again).

If you notice that detection doesnt work very well when far from the camera, you may want to decrease quad decimate by 1.

Use centers of trackers

By default, the trackers in VR will correspond to the main marker in the trackers IRL. This option will make them correspond to the center instead, which may reduce some of the shaking of trackers. It is recommended to try this option!

Search window

This parameter defines the size of the blue circle around trackers. If the circle is too large, detection will be slower. If the circle is too small, your trackers may move outside of the circles on movement, which will break detection. The default value is usualy a good balance, but you may want to make it smaller in order to get ATT working a bit faster.

Camera latency

The first thing you want to get correctly is the camera latency. Mostly necesary for IP Webcam (or other wifi connected camaras) as the delay with those is bigger, but even USB cameras can have a milisecond or two of delay, which we want to account for.

The best way to check whether your camera latency is correct is to move the tracker slowly at a constant velocity, and observe how the RGB axis move with it on the out window - the axis represents the smoothers prediction. The axis should follow the center of the tracker perfectly: if it lags behind, camera latency is too low, and if it is ahead, the latency is too high.

A good starting value for USB webcams is 0.02, and for IP webcam is 0.1. After updating the parameter, simply save, there is no need to restart detection.

Smoothing time window

The main parameter for the smoother, this defines how many past values will be used for smoothing. The more you use, the smoother will the trackers move, but you get more delay. The default of 0.5 usualy works fine, but with good cameras you can go down to 0.2 to make trackers more responsive.

This is something that is completely up to your preference. Find the value (probably between 0.5 and 0.2) that works best for you!

Additional smoothing

The second parameter of the smoother, this one is more basic: instead of trusting each value completely we refer to: new value = smoothing * old value + (1-smoothing) * new value. This removes any sudden large movements, which helps remove shaking while stationary, but makes it less responsive when moving.

Again, this is up to your preference. Usualy, you can use a value of 0.7 to make tracking noticably smoother with basicaly no effect on delay. You can go and use something large like 0.9 if you dont expect to move much, or down to 0.5 if you want tracking to be more responsive.

Depth smoothing

You only want to use this if you use multiple cameras. With pose estimation of trackers, position left/right, up/down on the camera can be determined extremely accurately. Distance from the camera, however, is estimated from size of the markers, which includes far larger error. This is a problem with multiple cameras: if we have a pose from two cameras from two directions, we want to discard this information to get more accurate pose.

This is what this parameter tries to do. When estimating pose, we first check the smoothers prediction for the current frame. If the estimated pose has a less than Depth smoothing error from the predicted pose, then depth is discarded and predicted depth is used instead.

Good values are something like 0.1 or 0.2.

Smoothing parameter visualizations:

Below is a demonstration of how smoothing works. Note that this is sythetic data, so it doesnt represent the tracking completely, but it should explain the principles.

Smoothing time window:

The main smoother works by using x past values to make a linear motion model, and predict current position based on that. The images should explain how that works a bit better:

Using only 5 past values, it doesnt help much in this situation: there is just too much noise to remove.

Using 15 values, it looks a lot better. When we start or stop moving, the smoothing overshoots a bit, but quickly corrects itself, so its not very noticable in game.

Using 30 values, the motion is very smooth, but it deviates from the actualy movement by quite a lot, which can be a problem during gameplay.

In this example, we used number of paramters, while in ATT we use a time window: using a 30 fps camera, 30 values correspond to time window of 1, 15 to time window of 0.5, and 5 to 0.15. Depending on your setup and ammount of noise it has, you may have diffrent results.

Additional smoothing:

Additional smoothing is much simpler: each frame, the position moves towards the new position by the given factor. This makes position very smooth when stationary, but there is a delay during movement.

Using 0.5, slightly smoother with basicaly no effect on delay:

Using 0.7, significantly smoother with minimal impact on delay:

Using 0.9, extremely smooth when stationary but introduces huge delay when moving:

Again, your results may be diffrent based on your setup, but usualy 0.7 works great with minimal effect on performance.

Camera latency:

Most cameras, especialy ones connected over wifi, will have some latency. We try to avoid it by predicting the values a few frames into the future, using the Camera latency parameter.

A latency of 10 frames, which is 0.33s in case of a 30fps camera, is exagerated and far too much for any camera, but it is useful to show how it works. Due to using old data for preditctions, the smoother will take longer to respond to changes in movement and overshoot.

In the case of a more realistic value of 3, or around 0.1s, the effect is far less noticable.

Usualy, if you use a wired webcam, you want to use a value of 0.05 or so.