Goal of the meeting is to discuss and align thoughts related to OGC API deployment & configuration. Idea of this report is to use it as input for the final report

Agenda/Tasks:

• GeoHealthCheck is used to evaluate the availability and complience of deployed endpoints.
• Portainer is a tool to monitor running containers, for example to check the log file of a container
• Postgres + PGAdmin can offer data storage for various containers, to see how we can manage tiff files (webdav?)
• LDPRoxy has been merged, optimise logging, latest software with bugfix is not yet available, postpone
• GeoServer, review PR, try to merge
• Discuss data strategy

Various products offer opportunity to configure new services using a web api (ldproxy, geoserver). Using the web ui to configure services, xml files are created on the server. It's important to understand that such an approach typically conflicts with maintaining a configuration in github and activate it as part of the deployment process.

It could be an interesting experiment with 2 instances, one having the github and one having the web-ui approach. And see the benefits/limitations of both approaches.

Discussion topics

During the meeting various topics around SDI deployment have been discussed

Data management

A typical use case will be that a geonovum employee arrives with some shapefiles to be published. The shapefile can be deployed as part of the deployment (via github). An alternative route is to import the data into postgres. The data can then be used in various applications. Importing a shapefile into postgres requires direct connection to postgres or use the PGAdmin dump import. GeoServer (via GeoCat Bridge) has an option to upload a shapefile and import it to Postgres.

Steps

• Set up is a conversion to postgres
• Create a service configuration for the layer

The current deployment can be run locally on Linux and Mac easily, which is helpfull to test a new development before creating the Pull Request. However it may be the case that this does not easily work on Windows. On the other hand, maybe this is not a scenario, because the GeoNovum employee might update the configuration in git, and deploy it to the test environment, and use that as a test prior to moving the configuration to the production system.

Use of attached storage

Using attached storage is common for larger files with a focus on read access. Can we use it to store grids (tiff) or tile cache to make it accessible for various services. Using attached storage as backend for geopackage or postgres is not optimal on attached storage because of many concurrent requests and file locking. Attached storage is usefull for bulk downloads (inspire stored query). Read access to a tilecache can be usefull, but seeding is problematic, due to the number of write requests.

Load balancing

Current setup does not have scaling (it is possible using traeffik, but currently not set up). It could be relevant to set up load balancing, at some point because it has its own type of challenges.

Auto scaling as provided by for example Kubernetes, is not in scope for this experiment. Kubernetes generally requires a large hosting farm, such as azure, google.

Include geoserver in the experiment?

We had some discussion if we should include GeoServer in the experiment. GeoServer is known to have challenges in cloud environments. But it is not explicitely known which challenges those are, and if their are ways to move around them. That's why it is usefull to include it. Also because the software has a high adoption at Dutch data providers.

An aspect of GeoServer challenges in cloud is the complexity of its config files. The config files are designed around the web user interface which is commonly used to set them up, and heavily depends on relations identified by complex uuid strings. Running multiple geoservers along side requires to synchronise the config files over the instances. This gets extra challenging in case

GeoServer has a jdbcconfig community module which allows to store the configuration in a database. But that plugin is not an official status yet (low TRL). Initiatives exist to define a geoserver cloud native strategy, based around an event bus.

URL configuration

Products tend to include a configuration parameter to indicate the outside url in which the service is made available. This url is for example used in a getcapabilities response to indicate the endpoint of the service. In CI/CD environments this parameter is challenging, because it may vary based on how you access the service (via internal or remote). There is actually no need to persist this in a parameter, because the value is also provided by the x-forwarded-for header of the request. Mind that the gateway software should be set up to add the x-forwarded-for header to the request. GeoServer facilitates for example this use case, in the settings you can activate a setting: 'use header for proxy url'.

Logs handling

To set up a proper mechanism to persist and visualise (error) logs is an important aspect of a successfull SDI implementation. Logs can be evaluated to find the cause of a problem, or more general find aspects to improve on the implementation. 3 types of logs can be identified:

• Error logs (generated by the application)
• Usage logs (typically at the gateway level)
• (un)availability (and hardware monitoring) logs (usually managed by dedicated software (geohealthcheck, zabbix, nagios)

Most basic approach is to make the log files directly available, for example using a tool like portainer. Alternatively one can deploy the ELK stack (Elastic search, Logstash, Kibana), grafana, splunk or similar and import the logs. A problem can then easily be researched from these 3 contexts.