Libavcodec Compression - CESNET/UltraGrid GitHub Wiki
FFmpeg provides a huge collection of codecs for use. Some of them (but not all) may be used with UltraGrid.
When no compression is specified, MJPEG is used. You can, however, pick another compression like H.264 or HEVC. Selection of other codecs is available as well (i. a. ProRes, VP9 or AV1).
Basic, but still useful and computationally decent compression is MJPEG which can be used if invoked in a following way:
uv -t testcard -c libavcodec:codec=MJPEG
uv -t testcard -c libavcodec:codec=MJPEG:q=23 # use quality 23 (approximately default)
uv -t testcard -c libavcodec # MJPEG is a default compression
uv -t testcard -c libavcodec:encoder=mjpeg_qsv:cqp=80 # QSV mjpeg uses normal JPEG range [0..100]
Note: quality can be affected only by the "q" parameter. Other quality-affecting parameters like "bitrate" are not honored.
In contrary to HEVC and AV1, H.264 supports interlaced video so it could be the preferred choice for interlaced content.
Usage:
uv -t deltacast -c libavcodec:codec=H.264 <address> # use H.264
uv -t deltacast -c libavcodec:codec=H.264:bitrate=20M <address> # specifies requested bitrate
uv -t deltacast -c libavcodec:codec=H.264:subsampling=420 <address> # enforce 4:2:0 subsampling (may be needed for the stream to be decodable with VAAPI)
Using NVENC encoding (NVIDIA only):
uv -t deltacast -c libavcodec:encoder=h264_nvenc
Use CUVID (HW accelerated) decoder (see below):
uv -d gl --param force-lavd-decoder=h264_cuvid
Since HEVC is relatively new and still a bit demanding compression providing a great compression ratio, you may need to tweak things a bit to achieve optimal performance.
There are multiple encoders supporting HEVC encoding, namely libx265 and hevc_nvenc (and also hevc_qsv if available). While encoding HEVC is still a bit demanding, it is advisable to use the NVENC encoder (or QSV) to encode the stream:
uv -t deltacast -c libavcodec:encoder=libx265 <addr> # x265 is slower to encode but parallelizable on decoder
uv -t deltacast -c libavcodec:encoder=hevc_nvenc <addr>
uv -t deltacast -c libavcodec:encoder=hevc_nvenc:bitrate=15M <addr> # reduce bitrate to be easier to decode
uv -t deltacast -c libavcodec:encoder=hevc_nvenc:spatial_aq=1 # improves quality at the expense of higher
# complexity, HW decoder may be required
uv -t deltacast -c libavcodec:encoder=hevc_qsv:cqp=5000 # use QSV with CQP=5000 (usable range approx. 2000-8000)
Currently, the stream encoded by NVENC encoder isn’t much parallelizable by decoder, so you may want to force hardware decoder (please note that the decoder currently adds some 4 frames of latency!):
uv -d gl --param force-lavd-decoder=hevc_cuvid
AV1 is supposed to be more efficient than HEVC but also more performance demanding, mainly on the encoder.
Nothing special is required for the decompression, libdav1d is the preferred decoder.
There are multiple encoders - rav1e, libaom-av1 and libsvtav1. The later 2 are recommended since rav1e currently (2024) lags behind those 2 in terms of performance. libsvtav1 is usually faster but there are some exceptions, like ARM-based Macs.
libsvtav1 supports multiple rate-control modes with different properties: VBR is triggered by passing bitrate option without cbr/cqp; not recommended, increases latency, see below), CBR (cannot be selected by now, see also below), CRF (implicit recommended) and CQP (also OK). For both CRF and CQP, the bitrate can be specified which sets MaxBitRate.
Examples:
-
SVT AV1:
uv -t <cap> -c lavc:enc=libsvtav1[:crf=35]Use CRF as the rate control mode (recommended).
CRF=35 is implicit/default, range 0-63, the smaller value the better compression (higher bitrate)
-
SVT AV1 with CRF and bitrate:
uv -t <cap> -c lavc:enc=libsvtav1:crf=35:bitrate=3Malso uses CRF=35 with bitrate limited to 3 Mbps
-
(SVT AV1 with bitrate:
uv -t <cap> -c lavc:enc=libsvtav1:bitrate=3M)use VBR with 3 Mbps. Currently not recommended because (with current options) greatly increases latency. CBR is unfortunately not possible to set by now due to incorrect settings in FFmpeg.
setting cpb_props->max_bitrate to a non-zero is refused by SVT AV1 in CBR mode. max_bitrate is must be set to >0 otherwise the FFmpeg plugin doesn't set CBR (see the link few lines above) -
SVT AV1 with QP:
uv -t <cap> -c lavc:enc=libsvtav1:cqp=30 -
AOM AV1:
uv -t <cap> -c lavc:enc=libaom-av1:bitrate=3M
Besides H.264/HEVC, UltraGrid supports also few other codecs that may be used, eg.:
- VP8/VP9
- J2K - FFMPEG's implementation of J2K is however very slow
uv -t deltacast -c libavcodec <addr> # use default libavcodec codec (currently MJPEG)
uv -t deltacast -c libavcodec:codec=MJPEG <addr> # use MJPEG codec explicitly
uv -t deltacast -c libavcodec:help <addr> # prints available codecs/encoders (and decoders)
See also a separate page Hw. acceleration support.
UltraGrid offers some hardware compression accelerations. Encoders are always standalone, therefore specifying encoder=<name> is sufficient. For decoders there are on one hand also standalone decoders that may be used similarly but also so called hardware accelerations (currently implemented in UG only for Linux).
Please note that the HW decoders may not be able to decode arbitrary video. If decoder refuses to decode, it is useful to try to set encoder to encode 8-bit 4:2:0 YCbCr video. You can try -c libavcodec[other_opts]:subsampling=420, if it doesn't help. :disable_intra_refresh may also improve compatibility. Color depth should be ideally controlled at the source (eg. -t decklink:codec=UYVY).
HW accelerated encoding is toggled by selecting appropriate encoder, eg. hevc_vaapi or hevc_nvenc:
uv -t testcard -c libavcodec:encoder=hevc_amf (Windows only)
uv -t testcard -c libavcodec:encoder=hevc_nvenc
uv -t testcard -c libavcodec:encoder=hevc_vaapi (Linux only)
uv -t testcard -c libavcodec:encoder=hevc_qsv
uv -t testcard -c libavcodec:encoder=hevc_videotoolbox (mac only)
VAAPI/VDPAU accelerated decoding (if supported) can be toggled with following command:
uv -d decklink --param use-hw-accel
CUVID and QuickSync are regular, HW-accelerated decoders (can be listed with -c libavcodec:help). Usage:
uv -d gl --param force-lavd-decoder=hevc_cuvid
uv -d gl --param force-lavd-decoder=hevc_qsv
or if unsure which codec arrives, you can list more decoders:
uv -d gl --param force-lavd-decoder=hevc_cuvid:h264_cuvid
Note: Using a cuvid may introduce additional latency.
Note 2: QuickSync decoder may not support intra refresh feature, in that case use -c libavcodec:encoder=<enc>:disable_intra_refresh on encoder.
Prefer CUVID/QuickSync and similar decoders for higher bit depths because current UltraGrid VAAPI/VDPAU implementation supports only 8-bits decoding.
Note: If using VAAPI for decoding, set 4:2:0 subsampling on the encoder, otherwise decoding won't work, eg.:
uv -t testcard -c libavcodec:encoder=libx264:subsampling=420 <receiver>
| API | Encode | Decode |
|---|---|---|
| NVENC | Win/Linux | Win/Linux (CUVID) |
| QuickSync | Win/Linux ¹ | Win/Linux ¹ |
| VideoToolbox | Mac | Mac |
| VA API ² | Linux | Linux |
| VDPAU ² | Linux | Linux |
- ¹ recent Linux distributions are required (aka Ubuntu 20.10 or Debian 11)
- ² see also Hw. acceleration support
See also:
- https://trac.ffmpeg.org/wiki/HWAccelIntro
- https://en.wikipedia.org/wiki/Quicksync
- https://en.wikipedia.org/wiki/Nvidia_NVENC
UltraGrid parameters allow more fine-grained selecting of parameters, eg. converting R12L to 10 or 12-bit 4:4:4 YCbCr, options are:
uv -t testcard -c libavcodec:encoder=libx265[:rgb|:yuv][:depth=<d>][:subsampling=<s>]O
Or you may still leave the selection on UltraGrid but changing the precedence policy. You may prefer keeping color-space instead of bit-depth and subsampling (whichi is default). This may be useful in particular to preserve performance preventing doing potentially costly color-space transformations:
uv -t testcard -c libavcodec:encoder=libx265 --conv-policy cds # == [c]olor>[d]epth>[s]ubsampling
uv -d deckink --conv-policy cds # works on receiver as well
There are also original no longer recommended options lavc-use-codec at encoder and decoder-use-codec:
uv -t decklink:codec=R12L --param lavc-use-codec=yuv444p16le -c libavcodec:encoder=hevc_nvenc <recv>
uv -d decklink --param decoder-use-codec=R12L <sender>
Here the codec is one that is supported by the display. If you enter an unsupported one, decoder writes you available options. Or you can use a keyword help to see display supported codecs.
It is sometimes tricky to know which pixel formats are available and provided by the decoder. Fortunately it can be probed with UltraGrid, just run (for example for x264; don't forgot to press Ctrl-C to exit UltraGrid):
uv -t testcard -c libavcodec:encoder=libx264 --verbose=7 2>&1 | grep 'supported pixel formats'
You can also check what codec would go from decoder. This is only informative, UltraGrid can do a conversion if needed. But if you expect 12-bit and there are only 8-bits available, it would need more tweaking (another decoder or a card):
uv -t testcard -c libavcodec:encoder=libx264 -d dummy --verbose=7 2>&1 | grep 'Available output pixel formats:'
Note: the available output pixel formats may differ according to input stream properties, thus pass the parameters to encoder exactly.
Sometimes is easier to get an overview of device capabilities first.
To see capabilities of a graphic card, you can download and compile this tool (requires nv-codec-headers) and see output of nvencinfo and nvdecinfo for an list of HW supported features
git clone https://github.com/FFmpeg/nv-codec-headers.git
cd nv-codec-headers && make && sudo make install
cd ..
git clone https://github.com/philipl/nv-video-info
cd nv-video-info && ./autogen.sh && make
./nvencinfo
Also this matrix may be useful.
For VDPAU and VAAPI there exist functions vdpauinfo and vainfo to check capabilities