StorageManager - ObjectVision/GeoDMS GitHub Wiki

A StorageManager is part of the GeoDMS software used to read or write data from and to files or databases.

If a StorageManager is configured, data is

• read from the storage when requested in a view or calculation.
• written to the storage when a data item is updated (results of the expression are calculated).

Requesting data in a view triggers the GeoDMS to update items, as well as the submenu items Update Treeitem and Update Subtree in the GeoDMS GUI.

Since 6.045 (read) and 7.408 (write) we use the GDAL library to read data from and write data to multiple formats, additional to the GeoDMS StorageManagers that are also still useful in specific cases.

The following example shows the configuration of the dbf StorageManager for reading data from a region.dbf file.

unit<uint32> Table
:   StorageName     = "%SourceDataDir%/CBS/region.dbf"
,   StorageType     = "dbf"
,   Source          = "dbf example source"
,   StorageReadOnly = "True"
,   SyncMode        = "None"
,   SqlString       = "SELECT * FROM TestTable ORDER BY ID"
{
     attribute<int32> att;
     attribute<int32> att2 := att * att, DisableStorage = "True";
}

A StorageManager is configured by setting properties to a data item or its parent item.

• StorageName
• StorageType
• Source
• StorageReadOnly
• DisableStorage
• SyncMode
• SqlString

• vector data (Shape files)
• grid data (GeoTiff, ...)
• ASCII Files (csv)
• parquet
• XML
• dbf
• database (GeoPackage)
• GeoDMS own formats
  • MMD
  • FSS

Since GeoDMS 20.20.0 a gdal.vect table and a str parameter are read as an operator application: the table unit gets a key expression storage_read_table(...) that names the storage, the sql string, the layer and every stored attribute, and each stored attribute is a member of that result. The consequences a modeller can notice:

• Only the attributes that are actually requested are read; the other columns of the table are left alone, also when SyncMode declared them. The attributes of a gdal.vect table that are requested together are read in one pass over its features.
• A read is scheduled like any calculation, so an IntegrityCheck on the table unit guards every attribute read from it, and a check may refer to the item it is configured on (see IntegrityCheck).
• The ExplicitSuppliers of a stored item are part of the identity of its read: they are calculated before the read and a change in them re-reads.
• The HasCalculator property of such an item reads True, while the GeoDMS GUI keeps showing it as a stored item (tree icon, DataSource row of the detail page) and refuses to edit it: its authentic value is in the storage.
• An item read from a storage is never written back to that storage, also when the storage is not StorageReadOnly.
• Two configured items that read the same table with the same attributes (and the same SqlString, ExplicitSuppliers and GDAL-Options) share one read and one result, wherever they are configured; their domains unify, so their attributes combine without an rjoin.

The same holds for every other storage manager. dbf, shp, odbc, xyz and FSS/cfs read a table unit and its stored attributes as one read (storage_read_table); an attribute that is not below its domain unit (the geometry of a shapefile, an FSS attribute over a unit defined elsewhere), a grid (GeoTiff, bmp, gdal.grid) and a strfiles or MMD attribute are read as one attribute over their configured domain (storage_read_attr), which for MMD means mapping the attribute's file as before. A unit of an MMD store keeps the range its dictionary declares and is not read.