The OGC Compliance Program provides the CITE tool for testing standards compliance. Our build process makes heavy use of this tool ... ensuring that each and every release of GeoServer correctly implements WMS, WFS, and WCS.

There are four problems with this:

• We last updated our CITE build tool 2012 (so our testing is quite out of date).
• geoserver.org website can only say we are a "compliant implementation", rather than a "certified implementation" of the WCS, WMS and WFS standards.
• New services such as WPS and CSW are not "blackbox" tested.
• Finally the transition to build.geoserver.org and subsequent rebuild means all CITE tests are currently failing

The OGC provide the CITE Team Engine both as a web service and as an open source project that can be used to check a single instance of GeoServer. This is a little awkward as we would like to integrate these tests into our build server; as such our build server stands up a GeoServer instance, performance the tests, and publishes the results here. These tests also require a database - and modify that database over the course of the tests (so our build server needs a PostGIS running).

Armed with a positive test result we can as an OSGeo Project ask for OGC certification, allowing us to update our website. Open source projects also have the option to be listed as a "reference implementation" (providing an instance of the server is available online for potential clients to test against).

Activities:

• 2017 Code sprint - design and approached discussed proof of concept of git submodule approach
• FOSS4G NA 2014 CITE Sprint - foss4gna activity to get developers set up to run cite tests locally

Resources:

• TEAM Engine
• geoserver-cite-tools - Justin's fork from 2013
• geoserver-cite-tools te-2015 - Andrea's
• geoserver data - cite data directory configurations

This activity is difficult enough that it has not attracted a contributor, and our previous 2012 CITE Test automation was good enough that no funding a replacement has been forthcoming.

This is intended as a sponsored activity:

• Gathering sponsorship from interested parties
• GeoServer Budget 2019 identified this as a priority

Propose the following:

1. A short term contract to setup the test harness and build scripts to automat cite tests on build.geosever.org.

• GSIP 176 CITE Automation RFP

2. A team code sprint to pass the tests once infrastructure support is in place

3. Work with OSGeo to achieve certification once tests results are available.

4. Work with OSGeo SAC or OGC to host GeoServer configurations to act as a reference implementation.

• Use git submodule to to manage CITE data directory configurations

• Setup CITE TEAM Engine as a submodule

  Control which revision exactly we run against using this approach. Avoid making our own fork and contribute any and all test fixes to the upstream project.

• Build support scripts

  • scripts to manage temporary GeoServer in build directory
  • scripts to manage CITE Test Database (assume PostGIS but hsql possible)

• Alternatives for running tests:

  • Option: Take advantage of the CITE TEAM Engine's new REST API to run tests

  • Option: CITE TEAM Engine may be available to run as a Docker container

  • Option: Run using a script (same as 2012 approach)

Setup a short-term contract, to set up the following:

• Upgrade the test harness

  • Use https://github.com/opengeospatial/teamengine as a git submodule

• Set up build jobs to test existing tests

  • WCS 1.0 - previously certified
  • WCS 1.1
  • WFS 1.0 - previously certified
  • WFS 1.1
  • WMS 1.1 - previously certified
  • WMS 1.3

See Restore CITE Certification.