TiledUnit - ObjectVision/GeoDMS GitHub Wiki

Unit functions TiledUnit

• TiledUnit(tilesize)
• TiledUnit(lower_bounds, upper_bounds)

The TiledUnit function results in a segmented/tiled domain unit. Two forms are supported:

Regular tiling: TiledUnit(tilesize) produces a tiled unit covering the full range of the source domain in regular blocks of the given size. The values unit of the tilesize argument must be the original domain unit from which the tiled unit is derived.

Irregular tiling: TiledUnit(lower_bounds,upper_bounds) produces a tiled unit where each tile is defined explicitly. The two attributes must share the same domain unit (with one entry per tile) and the same values unit. For each tile i, the range is defined by lower_bounds[i] (inclusive) and upper_bounds[i] (exclusive). Tiles produced this way are not required to be uniform in size or to cover the source range without gaps.

TiledUnits are often used for grid domains. For two-dimensional domains, the tilesize (or bounds) arguments are of a two-dimensional Point value type. For one-dimensional (tables) domains, they are one-dimensional. For one-dimensional domains the term segmented is also used as a synonym of tiled.

Tiled units serve two main purposes:

1. Memory efficiency: when the original domain has a very large number of items (think for example of a grid of 100 meter cells covering all of Europe), holding all data in memory at once is impractical. Tile-based processing keeps only the active tiles in memory.

2. Parallel processing: tiles can be processed independently across multiple threads, reducing wall-clock time on multi-core machines.

In modern GeoDMS both of these benefits are largely available without explicit TiledUnit configuration: domains use a default tiling (256 by 256 for grids, 65536 entries for 1D domains), and most operators automatically run as tile pipelines across these default tiles. For most workloads the implicit tiling is sufficient.

Explicit TiledUnit remains useful when:

• Tile boundaries should follow logical breaks in the data rather than uniform blocks (irregular form, see example 3).
• A specific tile size is needed to align with the layout of source data, for instance the block size of a tiled GeoTIFF.
• The default tile size is unsuitable for memory or scheduling reasons in a particular calculation.

When a tiled unit is configured as a domain unit, calculations are processed on each tile separately. The results are presented to the user in the same manner as untiled domains.

For background on GeoDMS internal tiling, the relation to source data tiling, and the performance implications, see GeoDMS Default Tiling.

• parameter or literal for tilesize
• attribute data items with a numeric or Point values unit for lower_bounds and upper_bounds

• For the regular form: the values unit of the tilesize argument must be the original domain unit from which the tiled unit is derived.
• For the irregular form: lower_bounds and upper_bounds must share the same domain unit and the same values unit.
• The number of tiles must not exceed 65536.

6.024

1: uint32 domain unit (regular tiling)

unit<uint32> buildings_untiled: nrofrows = 100000000;
unit<uint32> buildings := TiledUnit(25000[buildings_untiled])
{
   attribute<float32> a_attribute := union_data(., buildings_untiled/a_attribute);
}

result: a tiled/segmented domain of buildings, with tiles of 25000 entries each.

2: spoint (grid) domain (regular tiling)

unit<spoint> gridunit_untiled := range(gridset(
         point/Source/point_rd
        ,point_yx(-1f    , 1f     , point/Source/point_rd)
        ,point_yx(405600f, 111300f, point/Source/point_rd)
        ,spoint)
     ,point_yx(  0s,   0s)
     ,point_yx(500s, 400s)
   );

unit<spoint> gridunit := TiledUnit(point_yx(100s, 200s, gridunit_untiled));

result: a tiled grid unit with 10 tiles of 100 by 200 cells each.

3: irregular tiling (from the CRISP model)

In this pattern, each tile corresponds to a logical sub-area (for instance one continent or one country) rather than a fixed grid block. The bounds are stored as attributes on a separate tile-index domain:

unit<uint32> StudyAreaTile
{
   attribute<ProtoDomain_sub> LB_sub;   // lower-bound point per tile
   attribute<ProtoDomain_sub> UB_sub;   // upper-bound point per tile
}

unit<ipoint> domain_sub := TiledUnit(StudyAreaTile/LB_sub, StudyAreaTile/UB_sub)
{
   attribute<domain>     domain_rel     := mapping(., domain);
   attribute<domain_sub> domain_sub_rel := mapping(., domain_sub);
}

result: a 2D domain split into tiles whose extents follow the geometry of each study area. Tiles can have different sizes and need not tile a regular bounding box.

For two-dimensional regular tiling, sorted-value optimisations downstream are only available when the tiling has a single column (i.e. horizontal stripes of full width). Square tilings such as 256 by 256 prevent these optimisations but are typically the better choice for raster operations because they match the GeoDMS default tile layout and the block layout of well-tiled GeoTIFF source data.