Ecere GeoPackage Tiling Experiment 2 - sofwerx/cdb2-concept GitHub Wiki
CDB X Tiling/GeoPackage Experiment #2 (Ecere)
A series of GeoPackages of the San Diego CDB, plus a full resolution global coverage (for one particular month) of the NASA Visible Earth Blue Marble, along with an accompanying cdb.json
can be found at:
https://portal.ogc.org/index.php?m=projects&a=view&project_id=466&tab=2&artifact_id=95320 (11.3 GB)
The version currently online separates layers independently, each having their own settings for LOD grouping. For examples vector layers are each in a single GeoPackage ("groupLOD" : 0), while imagery is grouped by 7 LODs and elevation by 6 LODs. This is to demonstrate that a particular publisher could easily decide how best to organize the data to balance the number of files and the size of each GeoPackage, and a client understanding the LOD grouping would automatically be able to adapt based on the layer grouping.
A separate set of GeoPackages has also been produced where all layers for a particular LOD/tile have been organized together within the same GeoPackage. The cdb.json and the file listing for this is found at:
https://portal.ogc.org/index.php?m=projects&a=view&project_id=466&tab=2&artifact_id=95365
Both of these contain the points referencing the models (buildings and trees), but not the models / textures themselves [as of Experiment 2]. That will be covered in our next experiment. The San Diego 3D models are however available as 3D Tiles or binary glTF from our OGC API end-point at http://maps.ecere.com/ogcapi/collections/SanDiegoCDB:Buildings/ and http://maps.ecere.com/ogcapi/collections/SanDiegoCDB:Trees/ .
Inside the GeoPackages, all layers are tiled using the GNOSIS Global Grid (using TMS extension -- https://gitlab.com/imagemattersllc/ogc-vtp2/-/blob/master/extensions/14-tile-matrix-set.adoc). The level 0 GeoPackages for BlueMarble imagery will still work e.g. in QGIS, as there is no variable widths at level 0.
The imagery GeoPackages use JPG, while the terrain uses GeoTIFF and the gridded coverage extension (http://docs.opengeospatial.org/is/17-066r1/17-066r1.html).
The vector GeoPackages (Roads, Hydrography, as well as Buildings and Trees referencing 3D models) are stored in this experiment using Mapbox Vector Tiles. The vector tiles extensions are used for this ( https://gitlab.com/imagemattersllc/ogc-vtp2/-/blob/master/extensions/1-vte.adoc, https://gitlab.com/imagemattersllc/ogc-vtp2/-/blob/master/extensions/2-mvte.adoc, https://gitlab.com/imagemattersllc/ogc-vtp2/-/blob/master/extensions/4-vtae.adoc). Alternatively, regular vector layers storing the geometry as WKB in the features table could be used.
One thing that was noted is that even if the vector geometry is tiled and organized into multiple GeoPackages, it might be useful to support storing the data attributes separately only at the top-level (level 0) tiles, or in a single GeoPackage storing only data attributes, to avoid duplication of that information at each grouping level.
The next experiment will focus on 3D models -- for geo-specific, both one glTF for a whole tile, as well as individual models to be referenced by the points.
For the case of points referencing the 3D models (best suited for geo-typical), those glTF files would be stored in a single 3D models table, as well as a textures table (if the models share rather than embed textures).
For the geo-specific / one glTF for the whole tile, the glTF could potentially be stored in a tiles table instead, and the models constituting the payload (much like raster or vector tiles).
OGC API access demo
At this address: http://maps.ecere.com/ogcapi/collections/SanDiegoCDB
you can directly access the San Diego CDB data from our GNOSIS data store through our GNOSIS Map Server, including rendering maps, downloading coverages, accessing as tiles in different tiling schemes, accessing individual vector features, retrieving them as (re-merged) GeoJSON, visualizing them on GeoJSON.io and so on. To some extent, this demonstrates that even though the data is tiled, this layout actually supports a wide range of use cases.
At this address: https://maps.ecere.com/ogcapi/collections/SanDiegoLayers
you can directly access the prototype CDB X split GeoPackages. However at this point, our map server and visualization client do not present this as a unified data source, so instead the tiles structure and individual GeoPackages are individually accessible.
Visualization
We can currently visualize the individual CDB X/GeoPackage elevation and imagery directly in our 3D visualization tool. Like for the server however the split geopackages are not yet unified as a single data source.
Cesium JS / 3D Tiles demo
If you head over to https://sandcastle.cesium.com/ and copy/paste the following 4 lines of JS code:
var worldTerrain = Cesium.createWorldTerrain({requestWaterMask: true, requestVertexNormals: true, });
var viewer = new Cesium.Viewer("cesiumContainer", { terrainProvider: worldTerrain });
var scene = viewer.scene;
var tileset = scene.primitives.add(new Cesium.Cesium3DTileset({ url: "https://maps.ecere.com/ogcapi/collections/SanDiegoCDB:Buildings/3DTiles/tileset.json" }));
then click "Run", and zoom in onto San Diego, you should start seeing 3D buildings. However some buildings sometimes fail to appear, which may have something to do with listing tiles which are then returned empty and considered invalid by CesiumJS.
This 3D Tiles distribution (for OGC API collection https://maps.ecere.com/ogcapi/collections/SanDiegoCDB:Buildings) is currently being generated on the fly from our GNOSIS Data Store / E3D models. No textures just yet [as of experiment 2].
Once we have added the 3D models to the CDBX GeoPackage it should be possible to stream as 3D Tiles straight from that as well (the CDBX/GeoPackage, rather than this SanDiegoCDB).
Next steps
- A table for storing glTF models, for referenced 3D models (either only for geo-typical trees, or even the geo-specific buildings as well)
- Single glTF models covering a whole tile for geo-specific models
- Support for visualizing the dataset including 3D models in our GNOSIS Cartographer client
- Support for GNOSIS Map Server streaming 3D models from CDB X/GeoPackage
These further steps may be beyond the scope of what we can achieve during the CDB X sprints:
- Attribution per model within the single tile model. We support this directly in E3D for triangular face-level attribution (it was clarified that glTF2 does not supports this, and extensions were considerign vertex rather than face attributions).
- Support for unifying split GeoPackages as a single data source