Service: EasyVMAF - EyevinnOSC/community GitHub Wiki
EasyVMAF (serviceId eyevinn-easyvmaf-s3) runs VMAF video quality analysis on files stored in an S3-compatible bucket. In Eyevinn Open Source Cloud (OSC) it is a job: the instance runs one analysis from the command-line arguments you give it and then exits. License: MIT.
- Compute a VMAF score for a distorted (encoded) video against a reference (original) video, both read from S3-compatible storage.
- Write the result as a JSON file back to S3.
- Choose the VMAF model: default HD,
--model 4K, or--phone. - Work with AWS S3 or S3-compatible storage such as MinIO (see Service: MinIO).
- Validate transcoded renditions against the source before publication. VMAF (Video Multi-Method Assessment Fusion) is Netflix's open source perceptual quality metric.
Prerequisite: a bucket with the reference and distorted video files, for example a MinIO instance on OSC (minio-minio).
- Store the S3 credentials as service secrets with
create-service-secret(serviceIdeyevinn-easyvmaf-s3), for exampleaccesskeyidandsecretaccesskey. - Use
get-service-schemato confirm the options, thencreate-service-instancewithserviceId: "eyevinn-easyvmaf-s3". The instance name must be alphanumeric, at most 20 characters.
{
"name": "myvmafrun",
"AwsAccessKeyId": "{{secrets.accesskeyid}}",
"AwsSecretAccessKey": "{{secrets.secretaccesskey}}",
"S3EndpointUrl": "https://<your-minio-endpoint>",
"cmdLineArgs": "-r s3://videos/reference.mp4 -d s3://videos/distorted_1080p.mp4 -o s3://videos/distorted_1080p_vmaf.json"
}The job runs the analysis and exits. Use get-instance-status, describe-service-instance or get-logs-for-instance to follow progress. The output file appears at the -o location when the analysis is complete.
The same example with the OSC CLI:
npx -y @osaas/cli create eyevinn-easyvmaf-s3 myvmafrun \
-o AwsAccessKeyId="{{secrets.accesskeyid}}" \
-o AwsSecretAccessKey="{{secrets.secretaccesskey}}" \
-o S3EndpointUrl="https://myminio.minio-minio.auto.prod-se.osaas.io" \
-o cmdLineArgs="-r s3://videos/reference.mp4 -d s3://videos/distorted_1080p.mp4 -o s3://videos/distorted_1080p_vmaf.json"| Option | Required | Type | Meaning |
|---|---|---|---|
| name | Yes | string | Name of the instance. Alphanumeric, at most 20 characters. |
| cmdLineArgs | Yes | string | Command-line arguments passed to the easyvmaf_s3.py script: -r REFERENCE_INPUT -d DISTORTED_INPUT -o OUTPUT [--phone] [--model MODEL]. See the argument table below. |
| AwsAccessKeyId | No | string, sensitive | Access key ID for the S3 bucket (or S3-compatible service) holding the reference, distorted and output files. Passed to the tool as AWS_ACCESS_KEY_ID. |
| AwsSecretAccessKey | No | string, sensitive | Secret access key paired with the access key ID. Passed as AWS_SECRET_ACCESS_KEY. |
| AwsSessionToken | No | string, sensitive | Temporary AWS session token, for use with temporary security credentials together with the key pair above. |
| S3EndpointUrl | No | string | Custom S3 endpoint URL for S3-compatible storage such as MinIO, for example https://minio.example.com. Passed as S3_ENDPOINT_URL. If not set, the default AWS S3 endpoint is used. |
The credential options are secrets: create them with create-service-secret and pass them as {{secrets.NAME}}, which only works as the entire field value.
| Argument | Required | Description |
|---|---|---|
-r <s3-url> |
Yes | S3 URL of the reference (original) video |
-d <s3-url> |
Yes | S3 URL of the distorted (encoded) video |
-o <s3-url> |
Yes | S3 URL where the JSON result is written |
--phone |
No | Use the VMAF phone model instead of the default HD model |
--model HD|4K |
No | HD (default) or 4K VMAF model |
Example comparing two 4K renditions:
{
"name": "vmaf4k",
"AwsAccessKeyId": "{{secrets.accesskeyid}}",
"AwsSecretAccessKey": "{{secrets.secretaccesskey}}",
"S3EndpointUrl": "https://<your-minio-endpoint>",
"cmdLineArgs": "-r s3://videos/source_4k.mp4 -d s3://videos/transcoded_4k.mp4 -o s3://videos/result_4k_vmaf.json --model 4K"
}The output is a JSON file with per-frame VMAF scores and an aggregate score:
{
"VMAF score": 94.32,
"frames": [
{ "frameNum": 0, "metrics": { "vmaf": 96.1 } }
]
}A score of 100 means the distorted video is perceptually identical to the reference. Typical thresholds for streaming delivery are 85 to 95 depending on the use case.
- easyVmaf on GitHub, the upstream Python tool
- EasyVMAF S3 on GitHub, the OSC Docker wrapper
- Service: MinIO, S3-compatible object storage on OSC