Addition of the 2.0 version of the of the Web Feature Service.

Justin Deoliveira

2.2.0 release.

Under Discussion, In Progress, Completed, Rejected, Deferred

The primary motivation is simply to continue to evolve GeoServer as an OGC Web Feature Service implementation. Furthermore are mandated initiatives like INSPIRE that are mandating that organizations implement WFS 2.0.

As expected the entire patch is quite large. To facilitate easier review of the patch it has been broken up into multiple parts, along with the final merged patch.

• base (individual, merged) - Required changes to core modules
• soap
  • SOAP specific changes
• wfs
  • Everything else, changes to WFS module and dependent modules, output format extensions, etc… (patch)
• merged
  • The entire patch

The OGC service architecture in GeoServer is well defined and WFS 2.0 will follow suit. When implementing a new version of a service there is the choice to made of whether a new service class is required, or if an existing class can be modified to handle the new version. An example of the former is WCS 1.0 > 1.1, an example of the latter is WFS 1.0> 1.1.

For WFS 2.0 an in between approach is taken. While WFS 2.0 is different enough from WFS 1.1 in terms of schema, new operations, etc… to warrant a new class, at the same time we want to reuse all the current logic of the WFS 1.0/1.1 operations.

So in general the strategy is to implement new classes for a new service, but have the existing operations like GetFeature, etc… work against both sets of classes.

With the 2.0 service implementation comes a new request object model. The model is generated directly from the xml schema via EMF, similar to the WFS 1.0/1.1 model. This presents us with a problem: the existing operations are written against the old EMF request object model.

While there are a number of ways of handling this the current implementation takes the approach of coming up with a unified (version independent) request object model. This model is actually a wrapper around the individual object models for the specific versions. As a simplified example:

package org.geoserver.wfs;

class GetFeatureRequest {

  abstract String getVersion();

  class WFS11 extends GetFeatureRequest {
      net.opengis.wfs.GetFeatureType wrapped;

      String getVersion() {
         return wrapped.getVersion();
      }
  }

  class WFS20 extends GetFeatureRequest {
      net.opengis.wfs20.GetFeatureType wrapped;

      String getVersion() {
         return wrapped.getVersion();
      }
  }
}

This decision is influenced in large part by the XML parsing/encoding architecture. A large part of the work of implementing a new service is writing the code to parse and encode requests/responses to/from XML. However when we use an EMF model that is generated directly from the underlying xml schema as the service we save a lot of work as the geotools schema assisted parsing can automatically do most of the parsing and encoding for us, minimizing the manual work to write the XML bindings.

With the new unified wrapper request model in place, all the operations are written to work against it, rather than directly against the EMF request object models. This has the nice side affect of isolating from future protocol versions as well, as we can simply implement the wrappers/adapters for the new version without modifying the core operations.

WFS 2.0 brings with it a new version of the Filter encoding specification. Even though the filter schema is quite different (it now includes query) the end result is more or less the same as it was in WFS 1.1. However one notable addition is support for temporal filters which adds a new set of filters.

The implementation of this proposal will include the following support for temporal filters:

• A new set of geotools filter interfaces/classes
• XML parsing/encoding support
• In memory implementation

Native handling of temporal filters is out of scope. Although if time permits an attempt to implement native support in some JDBC datastores may be carried out.

WFS 2.0 adds the ability to page results of a GetFeature request via the startIndex and maxFeatures parameters. The WFS specification actually implies that a service must maintain consistent paging results in the light of transactions, but such a restriction is very difficult to implement and implies that the server be able to store large results in memory, or maintain database transactions between requests which goes against the stateless nature of WFS all together. For this reason we ignore this requirement in our implementation.

In order to properly do paging a back end datastore must also be implicitly capable of sorting. However most back ends (other than database backends) can’t do sorting. In these cases we simply ignore this and page through features in the order they are returned. Which means clients attempting to page through a set of features backed by a Shapefile will often get inconsistent results. The alternative is to raise an exception for non database backends. But after consideration it was decided that supporting paging for non database backends was worth it. Similar to how we support transactions for non database backends but only do them properly in that case.

Given that paging is a very useful add on to GetFeature support for paging will also be supported in the 1.0 and 1.1 WFS version as well.

WFS 2.0 also adds the ability to do joins in a GetFeature request. This has been implemented with much of the work being done at the geotools level. See this proposal for details .

Since joining can only be (easily) done efficiently with database backends, joins are only supported with database backends, and only joins among two features types backed by tables in the same database are supported.

WFS 2.0 also adds a new construct known as a stored query. A stored query is essentially a regular WFS query that is stored on the server and can be invoked by its identifier. The query can also contain parameters/placeholders that are filled in dynamically when the request is invoked. There are 4 new operations that come to manage stored queries:

1. ListStoredQueries
2. DescribeStoredQueries
3. CreateStoredQuery
4. DropStoredQuery

As an example, consider the following CreateStoredQuery request:

<wfs:CreateStoredQuery>
 <wfs:StoredQueryDefinition id='myStoredBBOXQuery'>
  <wfs:Parameter name='BBOX' type='gml:Envelope'/>  <wfs:QueryExpressionText language='urn:ogc:def:queryLanguage:OGC-WFS::WFS_QueryExpression'>           
   <wfs:Query typeNames='sf:PrimitiveGeoFeature'>
    <fes:Filter>
     <fes:ValueReference>pointProperty</fes:ValueReference>
     <fes:BBOX>      
      ${BBOX}
      </fes:BBOX>
    </fes:Filter>
   </wfs:Query>
  </wfs:QueryExpressionText>
 </wfs:StoredQueryDefinition>
</wfs:CreateStoredQuery>

The above creates a stored query named “myStoredBBOXQuery” which is a simple BBOX query in which the bounding box is parameterized. Invoking this stored query would look like the following:

<wfs:GetFeature>
 <wfs:StoredQuery id='myStoredBBOXQuery'>
  <wfs:Parameter name='BBOX'>
   <gml:Envelope srsName='EPSG:4326'>
    <gml:lowerCorner>57.0 -4.5</gml:lowerCorner>
    <gml:upperCorner>62.0 1.0</gml:upperCorner>
   </gml:Envelope>
  </wfs:Parameter>
 </wfs:StoredQuery>
</wfs:GetFeature>

Stored query definitions declare the “language” they are encoded in. The default and only mandatory language being “urn:ogc:def:queryLanguage:OGC-WFS::WFS_QueryExpression” which is essentially the query defined in the regular WFS dialect. This naturally makes stored query language a sensible extension point, allowing to define stored queries in other dialects. However for the time being stored query definitions will be left non extendible. Although the api internally has been written with extensions in mind.

When a stored query is created it is naturally persisted in the data directory. The directory structure chosen is to persist them is under the "wfs/storedqueries" directory . With the above stored query this would result in: <pre> <GEOSERVER_DATA_DIR> … wfs/ storedqueries/ myStoredBBOXQuery.xml

Many OGC services define optional SOAP bindings as an alternative to the regular OGC interface. SOAP support has been implemented as part of this proposal with much of the work being done at the OWS dispatcher level.

The dispatcher will now recognize when an OWS request is made wrapped in a SOAP envelope. An example request would be:

<soap:Envelope xmlns:soap='http://www.w3.org/2003/05/soap-envelope'>
 <soap:Header/>
 <soap:Body>
  <wfs:GetFeature>
   ...
  </wfs:GetFeature>
 </soap:Body>
</soap:Envelope>

When the dispatcher recognizes the SOAP request it does the following

• Set a flag on the Request object:

  class Request { ... /** * True if the request is a SOAP request. / public boolean isSOAP() { return soap; } /* * Flags/unflags the request as a SOAP request. */ public void setSOAP(boolean soap) { this.soap = soap; } ... }

1. Unwrap the “payload” of the SOAP envelope and pass the result into the normal OWS dispatching chain to be parsed/etc… by the regular xml readers.
2. Rewrap the encoded response in another SOAP envelope

The above works in cases where the underlying service is returning XML. For instance a response to a GetFeature SOAP request would look like:

<soap:Envelope xmlns:soap='http://www.w3.org/2003/05/soap-envelope'>
 <soap:Header/>
 <soap:Body>
  <wfs:FeatureCollection>
   ...
  </wfs:FeatureCollection>
 </soap:Body>
</soap:Envelope>

However if another content type such as an image is returned then the dispatcher needs more information to proceed. For example, WFS 2.0 mandates that a SOAP encoded DescribeFeatureType response be the xml schema for the feature type be encoded in base64. For cases such as these an additional interface named SOAPAwareResponse is implemented the Response object.

public interface SOAPAwareResponse {

    /**
     * Returns the value of the attribute of the "type" attribute to be included on the "Body"
     * element of the SOAP response.
     */
    String getBodyType();
}

Which essentially reports to the dispatcher how the body was encoded and included in the SOAP wrapping. For example:

<soap:Envelope xmlns:soap="http://www.w3.org/2003/05/soap-envelope">
 <soap:Header/>
 <soap:Body type="xsd:base64">
  <wfs:DescribeFeatureTypeResponse xmlns:wfs="http://www.opengis.net/wfs/2.0">
    PD94bWwg...YT4K
  </wfs:DescribeFeatureTypeResponse>
 </soap:Body>
</soap:Envelope>

FES 2.0 introduces the concept of an extended operator, which allows one to add new operators to the core filter language. This is very similar in nature to the purpose of a function. So the way extended operators have been implemented is to essentially make them syntactic sugar for filter functions.

Consider the following WFS request with a filter containing a function:

<wfs:GetFeature>
 <wfs:Query typeNames='sf:PrimitiveGeoFeature'>
  <fes:Filter>
    <fes:PropertyIsEqualTo>
       <fes:Function name="strMatches">
          <fes:ValueReference>name</fes:ValueReference>
          <fes:Literal>name-f002</fes:Literal>
       </fes:Function>
       <fes:Literal>true</fes:Literal>
    </PropertyIsEqualTo>
  </fes:Filter>
 </wfs:Query>
</wfs:GetFeature>

As an extended operator this request would look like:

<wfs:GetFeature>
 <wfs:Query typeNames='sf:PrimitiveGeoFeature'>
  <fes:Filter>
   <foo:strMatches>
    <fes:ValueReference>name</fes:ValueReference>
    <fes:Literal>name-f002</fes:Literal>
   </foo:strMatches>
  </fes:Filter>
 </wfs:Query>
</wfs:GetFeature>

Since a filter at the top level must return true or false it requires that the function being invoked be one that returns a boolean.

As with all new OGC service implementations the service must respond to a request with the latest version of the protocol in the case where a client does not specify a specific version. Technically this is only allowed for a GetCapabilities request but GeoServer is lenient in that it allows clients to do so for other operations. So clients looking to upgrade without specifying the 1.0 or 1.1 WFS version must be able to handle the new capabilities document, or new gml 3.2 based feature collection.

Andrea Aime: +1 Alessio Fabiani: Ben Caradoc Davies: +1 Gabriel Roldan: +1 Justin Deoliveira: +1 Jody Garnett: +1 Mark Leslie: Rob Atkinson: Simone Giannecchini:

JIRA Task Email Discussion Wiki Page