Duplicate Threshold Tuning - jhogsett/EMA-VFI-WebUI GitHub Wiki

Deduplicate Threshold Tuning - Test a range of duplicate detection thresholds

Why Use Threshold Tuning

  • Tuning allows running a series of Duplicate Frames Reports with increasing detection levels
  • Use it to find the best threshold for the Auto-Fill Duplicate Frames feature
  • For example, find the best threshold that:
    • Maximizes detecting real duplicates
    • Minimizes detecting false positives

How It Works

  1. Set Input PNG Files Path to a path on this server for the PNG files being tested
  2. Set Starting Detection Threshold to set the lower limit on tested thresholds
  3. Set Ending Detection Threshold to set the upper limit on tested thresholds
  4. Set Detection Threshold Increase Step to the increase in detection threshold after each round
  5. Set Maximum Duplicates Per Group to limit consecutive found duplicate frames
    • This can help to find a suitable upper limit for Ending Detect Threshold
    • Values:
      • Set to 0 to not limit the number of consecutive duplicates
      • Set to 1 to prevent duplicates altogether
      • Set to any other value to limit the consecutive duplicates
    • Tip: in most cases, duplicates will be limited to several frames only
  6. Click Create Report
    • The status of the tuning can be tracked in the progress bar
  7. When complete, the Download box will appear with a .CSV version of the report
    • Tip: The file is ready for import into spreadsheet software
  8. The Tuning Report box shows the result of the tuning series, or any error encountered

Tuning Report Columns

  • Threshold - the detection threshold for the tuning round
  • dupe_percent - the percent of duplicates found
  • max_group - the size of the largest found group of duplicate frames
    • Tip: The above columns are useful if graphing the results for analysis
  • dupe_count - the count of duplicates found
  • first_dupe - the frame index of the first found duplicate frame

Important

  • ffmpeg.exe must be available on the system path
  • The values for the Threshold sliders can be changed in the config.yaml file section deduplicate_settings

More Details

  • The FFmpeg mpdecimate video filter is used to detect and remove duplicates
    • The hi and lo mpdecimate parameters are set to the specified theshold
    • the frac mpdecimate parameter is set to 1
  • Example: a 30 FPS video with 24 FPS real frames (20% duplicated frames)
    • With Threshold set to the minimum 0, 1 frame was removed
    • When set to maximum 25000, all frames except 1 were removed
    • When set to the default 2500, 20% of the frames were removed