Layers and Data Sources - PIN-Cambodia/DMIS GitHub Wiki

General

Supported types

The DMIS app supports different types of data. The application can display ArcGIS REST, GeoJSON and WMS layers.

ArcGIS REST

Layers can be setup to point to an external ArcGIS REST service. At the moment only one ArcGIS REST layer is setup and can be identified (Pacific Disaster Center). Legend images are created using the ArcGIS REST service as well. Because only one icon needed to be displayed for the Pacific Disaster Center, any future additional ArcGIS REST layers will also only have one icon.

GeoJSON

GeoJSON layers are styled using a style definition documented here. The data is stored in the database table dmis_data. The data_source column in dmis_data links to the column layer_source in the table dmis_layers. Layer legends are generated in the app using the style definition.

WMS

These layers are setup and styled on GeoServer. Layer legends come from GeoServer as well.

Adding a layer

Adding a layer to the application is done by adding the layer to the database tables. For adding the data source and styling it, see the sections below.

Add a row to dmis_layers. This is where layers are stored that are available to the app.

layer_id is automatically generated
layer_name should match the GeoServer name if the layer is stored in GeoServer
map_category (UNKNOWN = 1, PREPAREDNESS = 2, INCIDENTS_WARNINGS = 3, ASSESSMENT_RESPONSE = 4)
layer_source is the source of the data
layer_type - layer types supported by the app are 'wms', 'geojson', 'arcgisrest'
layer_style is only populated with a style when it is styled on the client at the moment (so geojson layers)

Add layer info to dmis_layer_info
This is where layer information such as layer title and groups is stored in both English and Khmer. Add one row for each locale.

layer_id - this should match with the ID of the row that you have just added
locale - 'en' or 'km'
layer_title - this is the title that will be shown to the user
layer_group - the app orders the layers by group on exact match so make sure there are no spelling mistakes
layer_copyright

Static data

Category: Preparedness

Data

Format: Shapefiles and raster

Source: People in Need (PIN)

Processing

The static data delivered by PIN was firstly pre-processed to explore all the information found in them. The format, size, attributes, coordinate reference system, features and any other extra information required for the processing step were summarized in an excel file called "data_review" (T:\Operations\Projects\1859_PeopleInNeed\DMIS\Project Documents\data\data_review.xlsx). The data was also displayed in QGIS software making sure all the layers were displaying their attributes and coordinate reference system correctly.

The prioritized data by PIN are enumerated below:

Geographic borders: formed by layers Admin Boundary 0, Admin Boundary 1, Admin Boundary 2, and Admin Boundary 3 (the last 3 were merged in one single layer in the app).
Population: population data was provided classified for males, females, and also categorized by age. It was then decided to use the population density (raster file) to represent the distribution of the population along the country.
Number of households: this data was not provided but it was represented by the family attribute found in the layer named “Population Com CBD”.
Health Facility locations: there were 4 layers (Health Posts, Health Centre, Referral Hospital, and Operational District (OD) Hospital) providing different categories of health centres as well as their locations. All of them were displayed.
Evacuation Sites: the evacuation sites were provided in 3 different layers, one of them enclosing the information of all the evacuation centres found in the other 2, which corresponded to two different provinces.

Secondary data uploaded for the project were:

Roads (named as “rdsl”).
Dams.
Flood Extent taking place next to the hydrological sources (lakes, rivers, and dams).
Rivers and Lakes.
Tributaries.
Total population: total population for males and females were added from the layer named “Population Com CBD” and displayed.
Cambodia settlements. Data was downloaded from https://www.humanitartianresponse.info/. The data consisted of four shapefiles comprising of Province Towns, District Towns, Communes and Villages. The layers were combined into one shapefile, and a 'TYPE' field added to describe the settlement type. These types were:'PROVINCE', 'DISTRICT', 'COMMUNE' and 'VILLAGE'. More information regarding the data can be found at (T:\Operations\Projects\1859_PeopleInNeed\DMIS\Project Documents\data\DMIS\settlements\Cambodia Settlements Data.docx).

Visualising

The styling of the layers were done initially in QGIS to make some trials to show the customer and to upload them in to the Geoserver using the Database (DB) Management and PostGIS in QGIS. In order to find all the layers uploaded and styled, it is only required to use the word “Cambodia” in the SEARCH box, and all the static layers will appear.

The final styling was done in Geoserver, using the Styled Layer Descriptor (SLD) language and the YSLD occasionally. The styling for population density, roads, dams, flood extent, lakes and rivers, canals, tributaries, and administration boundaries layers were selected from the MangoMaps offer by the WFP Cambodia Interactive Mapping Platform (https://mangomap.com/chanvibol/maps/69179022-f3a2-11e4-8120-22000b368f7c/Humanitarian-Response-Forum--HRF----Cambodia:-CODs-and-FODs#). The rest of the layers were styled following the customer desire.

The styles used for all the layers are found in the “Style” subsection of “DATA” section. It will be enough to search the word “Cambodia” to find all the styles used for each layers.

Adding to app

See instructions for adding a layer. The layer type should be 'wms' in the dmis_layers database table and the layer source should be set to the GeoServer layer name.

Pacific Disaster Center

Category: Incidents and Warnings

Data

Format: ArcGIS WMS

Source: http://arcgis.pdc.org/arcgis/rest/services/global_public/pdc_integrated_active_hazards/MapServer

Processing

No processing is required.

Visualising

Because this is an external service, no styling is required.

Adding to app

See instructions for adding a layer. The layer type should be 'arcgisrest' in the dmis_layers database table and the layer source should be set to the REST endpoint.

Weather (Earth Networks Lightning)

Category: Incidents and Warnings

Data

Format: CSV

Source: S3 - Cambodia lightning data is supplied by Earth Networks who push lightning strike data to S3 every 5 minutes. Data is stored in the tw-dmis bucket in S3.

Processing

The CSV data is transformed into a geojson document and stored in the database. The transform process is managed by the following service which is documented: earthnetworks_service.py

Visualising

The vector data is styled using a style definition which supports basic styles. This style is stored in the database in the dmis_layers table and is sent to the app. The app then converts this style into an OpenLayers style.

Adding to app

See instructions for adding a layer. The layer type should be 'geojson' in the dmis_layers database table and the layer source should be set to the name of the data source that is stored in the DB.

NCDM

Category: Incidents and Warnings and Assessment and Response

Data

Format: ESRI JSON

Source: ESRI JSON files supplied by PIN

Processing

PIN uses an API to push ESRI JSON data to the database - POST /api/v1/data/{data_source}. The data can be updated by pushing a new version of the data to the API. A new row will be created in the DB (table dmis_data) to store the data and another API will return the most recent version of the data to the app. Only certain data_sources are accepted for NCDM layers at the moment. The POST /api/v1/data/{data_source} API will have to get updated if a new data source needs to get added.

Supported data sources:

ktm_pcdm_affected_healthcenter
ktm_pcdm_affected_road
ktm_pcdm_affected_school
ktm_pcdm_affected_wells
ktm_pcdm_at_risk_commune
ktm_pcdm_at_risk_village
ktm_pcdm_data_daily_actual (this is the data source for multiple layers)

In the API the ESRI JSON is converted to GeoJSON which is sent to the app.

Visualising

Adding to app

See instructions for adding a layer. The layer type should be 'geojson' in the dmis_layers database table and the layer source should be set to the name of the data source that is stored in the DB.