Service: EasyVMAF - EyevinnOSC/community GitHub Wiki

EasyVMAF

EasyVMAF (serviceId eyevinn-easyvmaf-s3) runs VMAF video quality analysis on files stored in an S3-compatible bucket. In Eyevinn Open Source Cloud (OSC) it is a job: the instance runs one analysis from the command-line arguments you give it and then exits. License: MIT.

What it can do

  • Compute a VMAF score for a distorted (encoded) video against a reference (original) video, both read from S3-compatible storage.
  • Write the result as a JSON file back to S3.
  • Choose the VMAF model: default HD, --model 4K, or --phone.
  • Work with AWS S3 or S3-compatible storage such as MinIO (see Service: MinIO).
  • Validate transcoded renditions against the source before publication. VMAF (Video Multi-Method Assessment Fusion) is Netflix's open source perceptual quality metric.

Creating an instance

Prerequisite: a bucket with the reference and distorted video files, for example a MinIO instance on OSC (minio-minio).

  1. Store the S3 credentials as service secrets with create-service-secret (serviceId eyevinn-easyvmaf-s3), for example accesskeyid and secretaccesskey.
  2. Use get-service-schema to confirm the options, then create-service-instance with serviceId: "eyevinn-easyvmaf-s3". The instance name must be alphanumeric, at most 20 characters.
{
  "name": "myvmafrun",
  "AwsAccessKeyId": "{{secrets.accesskeyid}}",
  "AwsSecretAccessKey": "{{secrets.secretaccesskey}}",
  "S3EndpointUrl": "https://<your-minio-endpoint>",
  "cmdLineArgs": "-r s3://videos/reference.mp4 -d s3://videos/distorted_1080p.mp4 -o s3://videos/distorted_1080p_vmaf.json"
}

The job runs the analysis and exits. Use get-instance-status, describe-service-instance or get-logs-for-instance to follow progress. The output file appears at the -o location when the analysis is complete.

The same example with the OSC CLI:

npx -y @osaas/cli create eyevinn-easyvmaf-s3 myvmafrun \
  -o AwsAccessKeyId="{{secrets.accesskeyid}}" \
  -o AwsSecretAccessKey="{{secrets.secretaccesskey}}" \
  -o S3EndpointUrl="https://myminio.minio-minio.auto.prod-se.osaas.io" \
  -o cmdLineArgs="-r s3://videos/reference.mp4 -d s3://videos/distorted_1080p.mp4 -o s3://videos/distorted_1080p_vmaf.json"

Options

Option Required Type Meaning
name Yes string Name of the instance. Alphanumeric, at most 20 characters.
cmdLineArgs Yes string Command-line arguments passed to the easyvmaf_s3.py script: -r REFERENCE_INPUT -d DISTORTED_INPUT -o OUTPUT [--phone] [--model MODEL]. See the argument table below.
AwsAccessKeyId No string, sensitive Access key ID for the S3 bucket (or S3-compatible service) holding the reference, distorted and output files. Passed to the tool as AWS_ACCESS_KEY_ID.
AwsSecretAccessKey No string, sensitive Secret access key paired with the access key ID. Passed as AWS_SECRET_ACCESS_KEY.
AwsSessionToken No string, sensitive Temporary AWS session token, for use with temporary security credentials together with the key pair above.
S3EndpointUrl No string Custom S3 endpoint URL for S3-compatible storage such as MinIO, for example https://minio.example.com. Passed as S3_ENDPOINT_URL. If not set, the default AWS S3 endpoint is used.

The credential options are secrets: create them with create-service-secret and pass them as {{secrets.NAME}}, which only works as the entire field value.

cmdLineArgs arguments

Argument Required Description
-r <s3-url> Yes S3 URL of the reference (original) video
-d <s3-url> Yes S3 URL of the distorted (encoded) video
-o <s3-url> Yes S3 URL where the JSON result is written
--phone No Use the VMAF phone model instead of the default HD model
--model HD|4K No HD (default) or 4K VMAF model

Example comparing two 4K renditions:

{
  "name": "vmaf4k",
  "AwsAccessKeyId": "{{secrets.accesskeyid}}",
  "AwsSecretAccessKey": "{{secrets.secretaccesskey}}",
  "S3EndpointUrl": "https://<your-minio-endpoint>",
  "cmdLineArgs": "-r s3://videos/source_4k.mp4 -d s3://videos/transcoded_4k.mp4 -o s3://videos/result_4k_vmaf.json --model 4K"
}

Using the instance

The output is a JSON file with per-frame VMAF scores and an aggregate score:

{
  "VMAF score": 94.32,
  "frames": [
    { "frameNum": 0, "metrics": { "vmaf": 96.1 } }
  ]
}

A score of 100 means the distorted video is perceptually identical to the reference. Typical thresholds for streaming delivery are 85 to 95 depending on the use case.

Resources

⚠️ **GitHub.com Fallback** ⚠️