Tools live under frame_gen/tools/ and support objective and subjective evaluation.

• Use Python 3.12.2

Setup:

python3.12.2 -m venv /home/user/venv/cgreplay_metrics
source /home/user/venv/cgreplay_metrics/bin/activate
pip install -r frame_gen/tools/requirements_tools.txt

1. PSNR and SSIM Analysis (frame_gen/tools/psnr_and_ssim.py)

• Calculates PSNR and SSIM
• Generates plots over time
• Usage: python frame_gen/tools/psnr_and_ssim.py

2. Real-time Quality Metrics Dashboard (frame_gen/tools/real_time_quality_metrics.py)

• Streamlit-based dashboard
• Calculates PSNR, SSIM, LPIPS, tLPIPS
• Usage: streamlit run frame_gen/tools/real_time_quality_metrics.py

3. Mean Opinion Score (MOS) Evaluation (frame_gen/tools/mean_opinion_score_video_pairs.py)

• Subjective QoE pipeline (randomized pairs, ratings/comments)
• Usage: python frame_gen/tools/mean_opinion_score_video_pairs.py

4. Video Utilities (frame_gen/tools/video_utils.py)

• Helpers for reading/writing/resizing/FR changes

Install libvmaf and FFmpeg with libvmaf support (Ubuntu 22.04/24.04):

Install dependencies:

sudo apt update
sudo apt install -y git build-essential pkg-config libtool \
    libssl-dev yasm cmake python3-venv meson ninja-build nasm \
    libass-dev libfreetype6-dev libtheora-dev libvorbis-dev \
    libx264-dev libx265-dev libnuma-dev

Build libvmaf:

git clone https://github.com/Netflix/vmaf.git
cd vmaf
make
sudo make install
sudo ldconfig
cd ..

Build ffmpeg:

git clone https://git.ffmpeg.org/ffmpeg.git ffmpeg
cd ffmpeg
./configure --enable-gpl --enable-libx264 --enable-libx265 --enable-libvmaf
make -j$(nproc)
sudo make install
cd ..

Verify:

ffmpeg -filters | grep vmaf

Expect lines like:

.. libvmaf           VV->V      Calculate the VMAF between two video streams.
.. vmafmotion        V->V       Calculate the VMAF Motion score.

Compute VMAF:

ffmpeg -i reference.mp4 -i distorted.mp4 -lavfi "[0:v][1:v]libvmaf=log_path=vmaf.json:log_fmt=json" -f null -

Tips:

• Align resolutions and synchronize frame counts.
• Persist per-frame metrics into CSV for downstream analysis.