Implementing a client application - adewg/ICAR GitHub Wiki

This page describes the ICAR ADE standard from a client perspective; what do you need to know to use the API's as described by the standard. You don't need to be a member of ICAR in order to use the standards (although there are many benefits to membership).

For more general information on ICAR and this standard, see About ICAR and the Animal Data Exchange Working Group.

Introduction

The standard consists of JSON-schema definitions defining the messages and an accompanying (recommended) URL scheme. The messages can be transported via many channels and thus can be used by themselves. We also provide a recommended URL scheme which allows REST-style access to these messages.

Most developers should be familiar with REST and JSON files. To simply view the standard, you can open the exampleUrlScheme.json file in your favorite OpenAPI Specification editor. For example, open it in the popular Swagger Editor. For a quick guide on getting started with code generation, see Generating code.

Backward and forward compatibility

When consuming the data, use a tolerant-reader pattern. If there are any fields in the message that you do not recognise, simply ignore them. This allows the standard to add new fields while maintaining backwards compatibility.

Synchronization with a service

There are use cases where a client has a requirement to know for certain that all events of a service have been retrieved. For example, for a milk recording agency it is essential to make sure all milking events have been recorded. This synchronization between a client and a service is not trivial and there are many ways to solve it. The ADE standard prescribes the following pattern to allow for synchronization.

Each event message has modified and source fields. The modified field is a datetime indicating the latest change to this messages record. The source field indicates the system of record for the message. These can be used for synchronization. All data services are required to fill these fields. The modified field is guaranteed to be monotonically increasing, e.g. it never goes back in time. This holds for each event related to the same source. A client thus may keep track of the latest received modified datetime per source and query all events from that time on. This guarantees that all changes will be retrieved.

There is a scenario that we do not cover in this specification: in case a device with a local clock has drifted too much, an operator may decide to reset its clock either forward or backward. If it is reset backward in time, clients may miss data recorded in that correction period. In case of such a hard reset, clients should be prepared to recapture a larger period in time. How to detect this is out of the scope of the spec and assumed a manual proces, similar to reparations needed with hardware failure or other kinds of data loss. In this scenario, the client should be able to rely on the ‘eventId’ being unique for the ’source’.

The url scheme should thus allow to query for meta-modified-from=x and meta-source=y. See Filtering resources for more details.

The WG also discussed future extensions: if messages (or rather, devices) are defined for which relying on a monotonically increasing datetime ('modified') cannot be implemented, the standard could define a tolerance period for those messages (e.g. it cannot go back in time more than 24hrs), or the standard could be extended with specific synchronization fields (for example, a specialized Sync group with sourceId and syncOffset allowing other synchronization methods). This will be considered when these messages or devices present itself with a use case.