How to edit a VFR(Variable Frame Rate) video avoiding audio sync issues in StaxRip - staxrip/staxrip GitHub Wiki

Editing and encoding a video that has a variable frame rate (VFR) or dropped frames, such as those shot by smartphones or recorded by video capturing utilities, etc., causes serious audio/video sync issues in many cases. This can be avoided in various ways depending on where the issue comes from.

The first step you can try is to stream-copy-remux your source file to mkv (in StaxRip or by ffmpeg) before loading (again) in StaxRip. This may fix the frame rate issue in some cases.

If this method does not help, try one of the following methods.

※ Note: Before you try the following methods, make sure to check if you have turned on the frame-rate auto-correction filters in Tools > Settings > Video and thus are unwittingly applying them.

This setting is helpful in many cases, but depending on the properties of the input video and what you want to do, it can sometimes obstruct your jobs.

MKV timestamps extraction

Starting in v2.7.0, timestamps can be set to be extracted from all or VFR MKV files. (Then they are muxed automatically in the output MKV file.)

For reference, timestamps extraction from all MKV files is made available to avoid the erroneous cases where the mediainfo is incorrectly recorded as CFR by some apps like OBS. See this issue thread for such a case.

If you are not changing the original frame rate, you can turn on this project option to extract the timestamps (=timecodes) of the source file and mux the timestamps file automatically in the output MKV.

If there is any change in the number of frames, frame rate, or duration, the Assistant will detect it and issue a warning/recommendation at the bottom of the main window. You can either follow its recommendation or reject it based on what you are trying to do.

Manual Conversion of Frame Rate

In many other cases, you can resolve the audio sync issue by converting the video frame rate to a constant one (CFR: constant frame rate) either at the source filter level or using filters that convert VFR to CFR based on the timecodes of the source file. You can use one of the following methods in StaxRip.

Source Filters

Select FFMS2 or FFVideoSource as the source filter. Open Code Editor and set the frame rate to a specific value by adding FPSnum and FPSden. For example,

FFVideoSource("%source_file%", cachefile="%source_temp_file%.ffindex", FPSnum = 30)

will set the frame rate to 30.

And

FFVideoSource("%source_file%", cachefile="%source_temp_file%.ffindex", FPSnum = 30000, FPSden = 1001)

will set the frame rate to 30000/1001 = 29.97 fps (NTSC).

You can right-click on the filter command and select a desired value from the fpsnum, fpsden menu.

You can do similar jobs with LWLibavVideoSource(LWLibavSource in VapourSynth) and LSMASHSource(LibavSMASHSource in VapourSynth). But since LSMASHSource uses a different library (L-SMASH) to demux a source file than the others (they are based on ffmpeg), it may fail in some cases where others don't.

In addition, you may use DSS2 if your video cannot be loaded with previous source filters in rare cases. It will automatically convert to CFR, but it requires DirectShow filters to be installed on your system. However, this is not a recommended method since DirectShow filters have several known issues in loading a video file. It's a fallback just in case.

Frame Rate Conversion Filters

TimecodeFPS, VfrToCfr in AviSynth, and VFRToCFR in VapourSynth are known to be able to convert VFR to CFR based on the timecodes of the source file. But there is a catch: they can be used only with FFmpegSource family source filters (FFMS2 or FFVideoSource) that can generate timecodes in the course of indexing the source file.

Limits and Caveat

Audio sync issues are one of the most intricate problems in AviSynth/VapourSynth. Since they can arise from many different causes, they need to be handled case by case in practice.

Frameserver-based processing (AviSynth and VapourSynth) seems to have this issue more often than ffmpeg because video and audio are separately processed in almost all cases while ffmpeg handles them concurrently based on timestamps. (AviSynth and VapourSynth focus on video frames based on the frame indexes generated by source filters.)

In almost all cases, following the above guide is usually enough.

However, you might run into some rare cases where the video frame rate variability is so high that after converting it to CFR, the audio becomes out of sync here and there in the middle of the video even if the start and end become synchronous. In those cases it may turn out that there is no alternative but to process the source using ffmpeg to avoid the issue.