HEVC Tile based adaptation guide - aureliendavid/gpac GitHub Wiki

HOME » MP4Box » DASH » HEVC Tiling

GPAC supports HEVC tile-based adaptation in DASH. This page gives a quick walkthrough on how to build such sequences.

Thanks to our friends at TUT, recent updates to the open-source Kvazaar encoder now allow anyone to encode HEVC with motion constrained tiling. In order to perform tile-based adaptation in DASH, the tiles must be constrained in motion prediction, and each tile has to be encapsulated in one slice (typical configuration).

Putting all your tiles in the same slice will not work, since the slice NALU(s) will likely not be split in different tracks, thereby forbidding individual downloads of tiles in DASH. In terms of Kvazaar encoding options, this means that you have to add to your usual command lines the following;

--tiles 3x3 --slices tiles --mv-constraint frametilemargin

This will instruct the decoder to use 3x3 uniform tiling for the encoding, with a single tile per slice in motion constrained mode. There are options in the Kvazaar encoder allowing you to specify a non-uniform tile grid, check the encoder usage/help.

You will need to package your raw HEVC bitstream with MP4Box, instructing it to split the tiles of each frame in different tracks, using the :split_tiles import switch:

MP4Box -add video_tiled.hvc:split_tiles -new video_tiled.mp4

For a NxM tiling, the resulting file will contain 1+NxM video tracks, a "base tile track" of type hvc2/hev2 containing parameter sets and/or SEI messages, if any, and NxM tile track of type hvt1 containing tile data.

You can check the motion constrained is well respected by removing a tile track from the file:

MP4Box -rem 3 video_tiled.mp4 -out test_tile_lost.mp4

It is not recommended to discard the first tile track, as playback may be broken depending on which version of the OpenHEVC decoder is shiped with your GPAC install (we will fix that soon). You should then see a video with a clean cut, green area in place of the removed tile:

Your video can be DASHed as any other video with MP4Box, for example

MP4Box -dash 1000 -profile live -out dash_tiled.mpd video_tiled.mp4

The MPD will contain as many adaptation sets as there are tile tracks and tile base tracks in the source file. Note that the current representation of tiled HEVC representations is still in discussion at MPEG and could change in the near future.

NOTE: If you want to perform tile-based adaptation, you will need to encode several motion constrained tiled videos at various qualities, and DASH them as usual. Note however that GPAC support for HEVC tiles adaptation is for now limited to videos encoded with the same configuration (same SPS/PPS/VPS), so do not  change your encoder settings too much except for the bitrate (especially, do not change the QP)  and always work with the same source video resolution/bit depth and tiling grid.

The resulting MPD will contain as many adaptation sets are there are tile tracks and tile base tracks in one of the input video, and each tile adaptation set will contain representations for each quality specified. One such example is available here.

You can now playback your MPD using GPAC, and have fun with the different adaptation modes for tiling (through the GUI or through the configuration file).

If you want to experiment with DASH adaptation for 360 video in GPAC, you will need to follow the above guide using an equi-rectangular projection video, and play the resulting MPD as follows:

MP4Client [-gui] DASH_URL#LIVE360

If you open a URL through the GUI, add #LIVE360 at the end of the URL. If you open a file through the GUI, long-click on the file and then select "View as 360".

In 360 navigation when the DASH TileAdaptation option is set to none, the DASH engine will select the tile quality based on the visible part of the 360 sphere.

NOTE: projection signalling in MP4 files or DASH MPD is still a hot debate at MPEG, this will likely be updated in the near future. We may also introduce other projection formats, such as cube maps.