What’s this about?

This tutorial describes how to upload a new dataset (ESRI shapefile) of IDPoor data to the DMIS portal. It assumes use of the IDPoor dataset (script to download data here) but the concept could be applied to any dataset.

What’s required?

• QGIS - https://qgis.org/en/site/forusers/download.html
• GeoServer Explorer Plugin for QGIS - https://plugins.qgis.org/plugins/geoserverexplorer/
• DBManager Plugin for QGIS - https://docs.qgis.org/2.14/en/docs/user_manual/plugins/plugins_db_manager.html
• Shapefile ready for publication.
• Optional - For doing more advanced database operations - PgAdmin (PostGres Admin) - https://www.pgadmin.org/

Helpful Links

PIN GeoServer (amazon ubuntu instance) http://ec2-13-250-26-33.ap-southeast-1.compute.amazonaws.com/geoserver/web/

Part 1 - Preparing Shapefile & Styling

• Load the Shapefile into QGIS in a new layer.
• Decide which field you want to symbolize the layer on. (i.e. Poor1HouseHolds, Poor1Poor2Households, etc.). Select a graduated symbol type. (Note, some symbology is restricted to datatypes. i.e. Graduated Symbology can only be used on Integers/Numbers)
• 'No Data' values should be established, in this case, the script enters '0' when no data is present. Otherwise the risk is that some data may not be displayed at all (i.e. it doesn't fit into a classification category)
• Right click the layer and select ‘Properties – Styles’

• At the top drop down, select ‘Graduated’ and choose a column to display with.

• Select a colour ramp from the list or manually set your own. Choose a number of classes and a mode. The most logical is ‘Natural Breaks’ which splits the data into ‘natural’ gaps in the data. You can play with other modes that may be more suitable, depending how you want to portray the data. Add a custom level for ‘No Data’ where the values are 0.

• Play with other symbolization options, transparency at 25% can look effective on the web map.

• Open GeoServer Explorer

• Create a new catalogue for the PIN DMIS Geoserver.

• Once connected, right click on the PIN Geoserver catalog and select Publish Layer

Important: Make a note the layer name – this will be entered into the database later and must match the name added to GeoServer.

• Now the layer is published in GeoServer. You can log into the Geoserver portal and check its working.
• Select Layer Preview to verify it is working.

Part 2 - PostGIS DMIS Database

Following Part 1, the layer is now live in GeoServer, but the DMIS app doesn’t know about that yet. We need to add the layer to the DMIS PostGIS Database so it appears on the map interface.

Two tables need to be altered. A function has been added which allows a single query to update both of these and link them with the appropriate ID. Note: There is no input for layer style as this is handled by QGIS back in Part 1.

_ Remember: The geoserverlayername parameter must match the exact layer name of the layer in Geoserver, this is how the map links the two together for display._

If required, the function definition can be found in the PostGIS database under Functions - dmis.newmaplayer

Call the function as below, all parameters must be entered (leave empty strings if not required).

SELECT dmis.newmaplayer( <geoserverlayername> <engthelayername text>, <mapcategory integer>, <layersource text>, <layertype text>, <layergeometry text>, <englayergroup text>, <engcopywrite text>, <kmthelayername text>, <kmlayergroup text>, <kmcopywrite text> )

A ready-2-go example; SELECT dmis.newmaplayer('idpoor_communes','ID Poor Households (Total Poor1+Poor2)',4,'geoserver','wms','polygon','Demographics','WFP','KhmerName','KhmerGrp','KhmerCW')

Executing the function

This function can be executed from whichever DBMS you like, but for ease of use, the DBManager Plugin works well with QGIS.

Create a new connection to the PostGIS database.

Then execute the call to function in the SQL window. If successful, the returned value will be the layer ID. Make sure the call is made on the dmis schema.

You should now see the new layer in the admin console on DMIS, as well as in the map layer switcher.

Known Issues / Workarounds

Styles

There can be some issues with styling, resulting in nothing displaying on the map, when in fact everything else is working fine. Use the 'layer preview' tab in GeoServer portal first, to ensure the data is actually up and running. It will display with a basic/generic style. Sometimes the styles simply don't match what you see in QGIS.

The styles in GeoServer are case sensitive. If the shapefile field is lowercase, i.e. P1P2HH but the style is lowercase i.e. p1p2hh - then the style will fail! They must be the same case. Sometimes, QGIS will convert field names to lowercase when importing, so this will break it.

Somewhere in the process, the style definitions can sometimes be broken. You can try manually updating the style file in GeoServer. It is an XML sort of code.

In QGIS, hit the Style button at the bottom of the Styles tab - Save as .SLD

This can be edited in a text editor as you wish. Then, manually upload it to GeoServer.

Sources & Further Reading

https://pypi.python.org/pypi/pyshp

-- End --