Filter Guide: Video Stabilizer - Crowsinc/LiveVisionKit Wiki

An ultra-fast video stabilizer used to smooth out video footage in real time.

The video stabilizer analyses the motion of video and smooths it out by counter-moving each frame. The frame must be cropped for it to move around without introducing any visible gaps in the video, so some content will be lost around the edges. Additionally, proper stabilization requires that the video content is suitable for camera motion estimation. This is detailed in the FAQ section.

Specifications

  • Filter Type: Hybrid video or effects filter
  • Performance Impact: High (>3ms)
  • Special Requirements:
    • Must have an OpenCL capable GPU.
    • Must be on Windows 10/11 to use as effects filter.
    • Must be applied before any HUD elements (donations, subscriptions etc.)
    • If used as an effects filter, you must manually delay audio to match the stabilized footage .

Settings

  • Smoothing Radius: How many past and future frames to consider for stabilization. Unfortunately time travel doesn't exist yet, so we must settle for introducing a video delay proportional to the radius. Higher values generally lead to smoother motion that is less reactive to camera movements. Consider setting this to its maximum value if the additional video delay doesn't bother you.

  • Stream Delay: Displays the video delay introduced by the stabilizer. The delay scales with the frame rate, so switching from 60FPS to 30FPS will double the delay. If the stabilizer is being used as an effects filter, you will need to manually delay the audio of the source by this amount. This is detailed in the FAQ section under Why is my audio out of sync?.

  • Crop: The percentage crop to apply to the video. The crop provides a hard limit on how much a frame can move for stabilization. Choosing a crop requires compromising between how much video resolution is lost and how large of a camera movement can be stabilized.

  • Motion Model: The mathematical model used for camera motion estimation and smoothing. The choices are:

    • Dynamic: Automatically chooses the motion model by making an educated guess on which one would be more suitable. This is not perfect, so you may want to consider experimenting with the other motion models to find which one is best for your type of content.

    • Affine: Restricted to only move, scale and rotate the frame. It is slightly faster and produces more realistic results in some scenarios. However is inadequate for most video and camera set ups, leading to jittery stabilization. Ideal for removing vibrations from a stationary camera.

    • Homography: Slower, but accurately moves and warps the frame as needed to achieve stabilization. This should be used by default.

  • Suppression Mode: Automatically suppresses (turns off) stabilization when the video content is considered unsuitable for stabilization. The choices are:

    • Off: Always stabilizes, regardless of its quality.

    • Strict: Only stabilizes when there is at least a 90% consensus in the estimated camera motion. That is, the stabilization turns off any time the scenery is not optimal. This is the recommended option for IRL content, where the stream content is dynamic and sensitive to bad stabilization. It may also be suitable for first person games that focus on exploring a 3D world.

    • Relaxed: Only stabilizes when there is at least a 30% consensus in the estimated camera motion. This is the recommended option for most game/VR content.

  • Disable Stabilization: Deactivates stabilization while retaining the specified crop and video delay. Used for seamlessly enabling and disabling stabilization without distracting viewers with the sudden application of the crop or pausing the video to reach the delay. Also useful for comparing the video with/without stabilization.

  • Test Mode: Aids in testing by drawing the following information to the screen:

    • Frame Time: The amount of time required to stabilize a single frame on your computer. To avoid stream lag, this number should be well below 16ms for a 60FPS stream and 33ms for a 30FPS stream.

    • Crop Rectangle: Displays the specified crop region.

    • Tracking Markers: Shows the tracking markers used for motion estimation. These will transition from green to red as stabilization is suppressed.

Frequently Asked Questions

  • What video content is suitable for stabilization? The stabilizer estimates the motion of the camera by analyzing the perceived motion that dominates each frame. For the camera to be stabilized, the landscape (which moves opposite to the camera) needs to be easily visible and must make up the large majority of the frame. Additionally, the range of depth within the frame should be fairly uniform. For example, if 50% of the content is very close to the camera while the rest is very far away. The stabilizer won't know whether to stabilize the nearby objects or the far away objects, which move faster relative to the camera. In terms of games, you will want to avoid any gameplay which has HUD elements, as these will be warped and moved around by the stabilizer. Suppression modes can help automatically disable stabilization when the video content is detected to be unsuitable. You may also use the 'disable stabilization' setting to manually stop stabilization when the content is unsuitable, while retaining the specified crop and video delay.

  • How do I pick which crop to use? The choice of crop depends on both your video resolution and the expected motion within the video. It is best chosen experimentally by running the stabilizer on old video content, and making a compromise between how large of a motion can be stabilized and how much resolution will be lost. At 1920x1080, a crop of around 3-4% is enough for removing most vibrations, while a larger crop of 8-12% may be necessary to handle aggressive swaying motions caused by walking or running. Motions which cannot be stabilized with the given crop will appear as sudden jerks in a previously smooth video.

  • I'm using a shoulder mounted camera, why does my body become shaky? When the camera is mounted on your shoulder, it is shaking in synchrony with your body movement. Meaning you are perceived to be stable relative to the camera, while the landscape is seen as being shaky. When stabilization is applied, the landscape is forced to become still by counter-moving it based on its perceived shake. You are assumed to be part of the landscape, so the counter-movement forces you to become shaky while the landscape becomes stable.

  • Why does my stabilization look jittery or like jello? Jittery stabilization is often the result of inappropriate use of the affine motion model; try forcing the stabilizer to use the homography motion model. The jello effect is caused by your camera's rolling shutter distortions, which become accentuated once stabilization is applied. This is best fixed at the source by configuring your camera to minimize rolling shutter distortions.

  • Why is my audio out of sync? If you are using the video stabilizer as an effect filter, you will need to manually delay your audio by the amount shown in the stream delay setting. One way of doing this is by adjusting the sync offset of the source via the Advanced Audio Properties screen. This can be accessed by clicking one of the cog buttons in the OBS Audio Mixer, then clicking the 'advanced audio properties' button.

  • My video won't stabilize no matter what settings I use, what can I do? If no combination of settings is able to produce a stabilized video, then your video likely cannot be stabilized by the LiveVisionKit stabilizer. You may want to try other existing stabilization solutions, or attempt to solve the problem at the source. This kind of stabilization isn't magic, there is currently no perfect solution and certainly not one that also runs at over 60FPS.