Annex C - STARIONGROUP/COMET-SDK-Community-Edition GitHub Wiki

ECSS-E-TM-10-25A Annex C specifies the implementation technology and protocols that are required to exchange data that complies with the data model specified in ECSS-E-TM-10-25A Annex A. Annex C prescribes 2 protocols that need to be supported by an implementation for it to be fully compliant with ECSS-E-TM-10-25A:

• JSON REST API
• JSON Exchange File Format

CDP4-COMET is fully compliant with the standard. The CDP4-COMET-SDK can be used to create applications client and server side that implement the JSON REST API. It also provides an implementation of to read from and write a file that conforms to the JSON Exchange File Format. The CDP4-COMET Desktop application can communicate with a JSON REST API, of which the CDP4-COMET Web Services are an implementation. The CDP4-COMET Web Services can export data to a JSON Exchange File Format and can be seeded from a JSON Exchange File Format.

CDP4-COMET provides extensions to both Annex A and Annex C.

• The Annex-A Extensions include extra classess to enricht the data-model.
• The Annex C Extensions specify additions to the protocol.

The JSON REST API is a particular flavor of REST. Two HTTP verbs are supported: GET and POST, DELETE, PUT and PATCH are not supported. GET requests are used to retrieve data. POST requests are used to create, update and delete data; multiple operations are supported in one POST request, allowing an API user to add, update and delete different objects with one request.

In order to retrieve data from a server a GET request must be performed. A GET request is performed along the containment hierarchy specified by Annex A. Annex A provides two so-called TopContainer classes, the SiteDirectory and EngineeringModel. The GET requests can only be performed on either the SiteDirectory and EngineeringModel URI's. The subsequent parts of the URI follow the aggregate composition properties of Annex A. To GET a specific Thing the unique identifier, represented by the {iid} token, needs to be specified as well. The {iid} token must be a valid UUID version 4.

The result of a GET request is JSON array of JSON objects that represent ECSS-E-TM-10-25 Annex A objects. The ClassKind property of the Thing class denotes its type and is used for serialization and deserialization purposes.

Annex C also supports query parameters. These query parameters are used to either narrow or widen the scope of the expected result. The following query parameters are supported and may be used in combination, unless specified otherwise:

NOTE: Even though the includeReferencaData query parameter is provided it is not recommended to use this when performing a GET request on an Iteration in an EngineeringModel. From the URI of the GET request it is possible to determine what Iteration instance the response is related to. This is useful when determining the containment tree in client applications of said data since the unique identifiers of objects contained in Iterations is not truly unique across the Iterations in an EngineeringModel. When Reference Data is returned as well it may become difficult to determine to what containment tree this data belongs, not so much for the reference data itself, but for concepts such as Alias, Definition and Hyperlink, these classes may be contained by data in the EngineeringModel containment tree and SiteDirectory Tree.

The CDP4-COMET Web API makes use of the POST verb to create, update and delete items. As such it is not a pure REST API since the PUT, PATCH and DELETE verbs are not supported. A POST request must be accompanied with a POST message that contains information on what items are created, updated and deleted.

The CDP4-COMET Web API accepts a POST message to the following routes:

http://hostname:port/SiteDirectory/{iid}
http://hostname:port/EngineeringModel/{iid}/iteration/{iid}

It is not possible to send POST messages to any other route, this includes an EngineeringModel route:

http://hostname:port/EngineeringModel/{iid}

Even though an EngineeringModel contains items other than Iterations, to create, update or delete these kinds of Thingss a, POST message must be sent to an Iteration that is contained by the same EngineeringModel.

A POST message is a JSON object that contains three properties also called parts; a part that contains those objects that need to be created, a part that contains the objects that need to be updated and those objects that need to be deleted. COMET® Web Services will execute the POST message as a transaction and if an operation on one of the instances fails, the whole operation or POST fails. As such the COMET® Web Services guarantee so-called transaction safety.

The post message has the following format:

{
  "_create": [],
  "_update": [],
  "_delete": []
}

The _create and _update properties of the transaction object (the POST message) are used to create new items and update existing items. Except for the TopContainers SiteDirectory and EngineeringModel, all the classes in the CDP4-COMET data-model are contained by another class. For instance a EmailAddress is contained by a Person. Every container has knowledge of those objects that it contains through the containment properties (composite aggregation), the contained items does not have knowledge of it's container. The result of this is that the creation of a new object also means an update on it's container. The POST message will contain the new object in the _create part, and an update to the containment property of it's container in either the _update part or the _create part. The object that is to be created must be complete, all the properties that are mandatory need to be included in the object.

Objects that are to be updated may be incomplete, only the properties of the object that are updated must be included in the _update part as well as the unique id and the classKind properties. The properties may be scalar or compound, where compound properties may be ordered or unordered. When a scalar property is updated the value of that property is replaced. When an unordered compound property is specified, the items in the collection are added to the object in the CDP4-COMET WebServices, provided that objects with those unique id's are also present in the _create part. If there are no contained objects in the _create part, the intention is to move the objects with those unique id's to the specified object, provided they exist.

The order of items in an ordered compound property is specified as a JSON object that has two properties: a key and a value. The key property is denoted with a k and the value property is denoted with a v. When the k and v properties are present in an object contained in the _update part, the key and value are added to the object in the CDP4-COMET WebServices.

{
  "k": "some key",
  "v": "some value"
}

The order of the items in an ordered compound property may be changed. This is achieved by adding an m property to the ordered item. The value of the k property is updated with the value of the m property in the CDP4-COMET WebServices. The v property must specified as well to assert that the item already exists in the ordered collection.

{
  "k": "original key",
  "m": "new key",
  "v": "existing value"
}

The _delete property of the transaction object in the POST message is used to delete Things from the CDP4-COMET WebServices. Both complete objects can be deleted or items can be removed from a compound property.

A complete object can be deleted by specifying only it's unique id iid and classKind property. The following example deletes the complete Person object.

{
  "_delete", [
    {
      "iid": "77791b12-4c2c-4499-93fa-869df3692d22",
      "classKind": "Person"
    }
  ],
  "_update": [],
  "_create": []
}

When an object with an unordered compound property is specified in the _delete part, the items in the specified property are deleted from that property in the CDP4-COMET WebServices. The following example deletes the UserPrefence object with unique id equal to 2999e725-9b19-46ec-85bf-e8d22914bad4 from the specified Person object.

{
  "_delete", [
    {
      "iid": "77791b12-4c2c-4499-93fa-869df3692d22",
      "classKind": "Person",
      "userPreference": "2999e725-9b19-46ec-85bf-e8d22914bad4"
    }
  ],
  "_update": [],
  "_create": []
}

When an object with an ordered compound property is specified in the _delete part, the items with the specified sort key(s) are deleted from the ordered compound property. Both the k and v property are specified, the v property is used to verify that the items exists for the specified k value in the ordered compound property. The following example deletes the ruleVerification object with sort-key 12345 and unique id b198ce8e-24c3-409b-ac6b-e0351de51f66 from the RuleVerificationList object:

{
  "_delete", [
    {
      "iid": "98d0c9ee-a98d-4ff0-b9b1-df8eab3a6c2f",
      "classKind": "RuleVerificationList",
      "ruleVerification": [
        { 
          "k": "12345",
          "v", "b198ce8e-24c3-409b-ac6b-e0351de51f66"
        }
      ]
    }
  ],
  "_update": [],
  "_create": []
}

The copy operation is available on CDP4-COMET Web-Services. The protocol is described here

Instances of particular classes can only be created and deleted by the CDP4-COMET WebServices and not by an application that is using the CDP4-COMET Web API. The creation these classes occurs as a side-effect of a create, update and/or delete of other classes. The following sections specify these side-effects.

It is forbidden to include an EngineeringModel in the _create or _delete part of a POST message. An EngineeringModel is created as a side-effect of a creation an EngineeringModelSetup

• create an EngineeringModel
• create an Iteration that is contained by the created EngineeringModel
• create an IterationSetup that is contained by the posted EngineeringModelSetup
• create a Participant that references the Person that has performed the POST request
• adds the default DomainOfExpertise to the activeDomain property of the EnineeringModelSetup of the Person that has performed the POST request

A Person that does not have a default DomainOfExpertise set may not create an EngineeringModelSetup When an EngineeringModelSetup is deleted the referenced EngineeringModel is deleted as well.

An EngineeringModelSetup can be instantiated from a source model by setting the sourceEngineeringModelSetupIid property to the Iid of the EngineeringModelSetup and the requiredRdl as blank when performing a Create POST. In this mode the web server will create/instantiate the requiredRdl (ModelReferenceDataLibrary) from the source EngineeringModelSetup by the server.

Depending on the EngineeringModelKindof the new and referenced source model respectively different instantiation modes exist. The following pseudo code provides the logic:

newKind = Kind of EngineeringModel that will be created from the request
sourceKind = Kind of EngineeringModel of referenced source EngineeringModelSetup

CopyAll = copy all data (incl all iterations)
CopyLatest = copy latest data only (only active iteration)
CreateTemplate = create template from referenced engineering model setup (study)

    // invalid instantiations:
    if (newKind == 'MODEL_CATALOGUE' && sourceKind == 'STUDY_MODEL')    
        -> throw 'May not create MODEL_CATALOGUE from STUDY_MODEL.'

    if (newKind == 'TEMPLATE_MODEL' && sourceKind == 'MODEL_CATALOGUE')
        -> throw 'May not create TEMPLATE_MODEL from MODEL_CATALOGUE.'

    if(newKind == 'SCRATCH_MODEL' && sourceKind == 'MODEL_CATALOGUE')
        -> throw 'May not create SCRATCH_MODEL from MODEL_CATALOGUE.'

    // determine instantiation mode by fallthrough logic:   
    -> CopyAll = false
    -> CopyLatest = false
    -> CreateTemplate = false

    if (newKind == 'STUDY_MODEL')
        -> CopyAll = true

    else if (newKind == 'TEMPLATE_MODEL' && sourceKind == 'STUDY_MODEL') 
        -> CreateTemplate = true

    else
        // newKind == 'MODEL_CATALOGUE' 
        // newKind == 'TEMPLATE_MODEL' 
        // newKind == 'SCRATCH_MODEL')
        -> CopyLatest = true

It is forbidden to include an Iteration in the _create or _delete part of a POST message. An Iteration is created as a side-effect of a creation of an IterationSetup. When an IterationSetup is created the CDP4-COMET WebServices will create an Iteration that is contained by the created EngineeringModel that is referenced by the EngineeringModelSetup that is the container of the IterationSetup.

When a Glossary is updated and the isDeprecated property is set to true, then all the isDeprecated of the Term instances contained by that Glossary will be set to true as well.

It is forbidden to include an ParameterValueSet in the _create or _delete part of a POST message. ParameterValueSet instances are created and deleted as side-effects of different create, delete and update operations:

• A ParameterValueSet is created when a Parameter is created. If the Parameter is not option dependent, nor state dependent then one ParameterValueSet is created. If the Parameter is option dependent then as many ParameterValueSet instances are created as their are Option instances in the Iteration that the Parameter is created in. If the Parameter is state dependent then as many ParameterValueSet instances are created as their are ActualState instances in the referenced ActualStateList. If the Parameter is both option and state dependent, then the amount of ParameterValueSet instances that are created is the combination of Option instances and ActualState instances.
• A ParameterValueSet is created when a Parameter is updated and set to be option and/or state dependent. The same rules apply for the creation of a Parameter
• A ParameterValueSet is created when an option dependent Parameter exists and an Option is added to the Iteration
• A ParameterValueSet is created when a state dependent Parameter exists and an ActualStateList is updated and references additional PossibleFinitestateList instances
• A ParameterValueSet is created when a state dependent Parameter exists and a PossibleState is added to the PossibleFinitestateList that is included in the ActualStateList that is referenced by the Parameter
• A ParameterValueSet is deleted when the container Parameter is deleted.
• A ParameterValueSet is deleted when the container Parameter is option dependent and is updated and set to be not option dependent.
• A ParameterValueSet is deleted when the container Parameter is option dependent and the referenced Option is deleted.
• A ParameterValueSet is deleted when the container Parameter is state dependent and is updated and set to be not state dependent,
• A ParameterValueSet is deleted when the container Parameter is state dependent and the referenced ActualState is deleted, see the section on Possible and Actual Finite States.

The values of the ParameterValueSet should be set as follows:

• When the ParameterValueSet is created as part of the creation of a Parameter, the values will be set to the default value -.
• For the other 4 ParameterValueSet creation cases mentioned above (when the Parameter already exists), the values of the ParameterValueSet are set using the values of the existing ParameterValueSet. If there are multiple existing ParameterValueSet, the values of the ParameterValueSet that corresponds to the **default ** (option, state or combination of both) one are used.
• When ParameterValueSet instances are deleted because the container Parameter is updated and set to be not state and option dependent, the values of the remaining ParameterValueSet, are set with the values of the ParameterValueSet that corresponds to the **default ** option, state (or combination of both).

It is forbidden to include an ParameterOverrideValueSet in the _create or _delete part of a POST message. ParameterOverrideValueSet instances are created and deleted as side-effects of different create, delete and update operations. A ParameterOverrideValueSet is created when a ParameterOverride is created. A ParameterOverride references the Parameter that it overrides. The side-effects specifed for the ParameterValueSet are valid for the ParameterOverrideValueSet as well.

It is forbidden to include an ParameterSubscriptionValueSet in the _create or _delete part of a POST message. ParameterSubscriptionValuSet instances are created and deleted as side-effects of different create, delete and update operations. A ParameterSubscriptionValueSet is created when a ParameterSubscription is created. A ParameterSubscription is contained by the Parameter or ParameterOverride that it subscribes to. The side-effects specifed for the ParameterValueSet and ParameterOverrideValueSet are valid for the ParameterSubscriptionValueSet as well.

It is forbidden to include an ActualState in the _create or _delete part of a POST message. ActualState instances are created and deleted as side-effects of different create, delete and update operations.

• An ActualState is created when an ActualStateList is created. As many ActualState instances are created as the combination of PossibleFinitestate Instances that are contained by the PossibleFinitestateList that are referenced by the ActualStateList.
• An ActualState is created when a PossibleFinitestate is added to a PossibleFinitestateList that is being referenced by an ActualStateList
• An ActualState is created when a PossibleFinitestateList is added to the PossibleFinitestateList that are being referenced by an ActualStateList. As many ActualState instances are created as new combinations of PossibleFinitestate instances can be created.
• An ActualState is deleted when the ActualStateList is deleted that is the container of the ActualState.
• An ActualState is deleted when a PossibleFinitestate is deleted from a PossibleFinitestateList that is being referenced by an ActualStateList.
• An ActualState is deleted when a PossibleFinitestateList is deleted from the PossibleFinitestateList instances that are being referenced by an ActualStateList.

The JSON Exchange File Format is used to exhange a Site Directory, Reference Data Libraries and Engineering Models between 2 parties. The CDP4-COMET Web Services can be initialized, or seeded, from such a file. The archive is an implementation neutral means to exchange ECSS-E-TM-10-25 data.

ECSS-E-TM-10-25 does not specify how Annex C.2 implentations should store the data, this is an implementation detail. The CDP4-COMET Web Services make use of a relational database called PostgreSQL. Other impementations may use a different kind of database, or different SQL schema.

The exchange file is a ZIP archive that contains multiple folders and files. These files contain ECSS-E-TM-10-25 objects that are serialized as a JSON array of JSON objects; the serialization is similar to serialization used in the response of a GET request as specified in Annex C.2. The following folders and files are contained in the ZIP archive:

• One Header File named Header.json, in the root folder;
• One SiteDirectory File named SiteDirectory.json, in the root folder;
• A SiteReferenceDataLibraries folder that contains zero or more SiteReferenceDataLibrary File;
• A ModelReferenceDataLibraries folder that contains zero or more ModelReferenceDataLibrary File;
• An EngineeringModels folder containing zero or more EngineeringModel Folder, where each EngineeringModel Folder contains:
  • One EngineeringModel File;
  • One Iterations folder that contains one or more Iteration File;
  • One FileRevisions folder that contains zero or more FileRevision File;
• Zero or one Extensions folder that may contain any application-dependent folders and files.

• The archive contains one Header File which is named "Header.json".
• The Header File is stored in the root folder of the archive.
• The Header File contains the following information, in the form of a single JSON object with the following string or null valued properties:
  • mediaType: fixed identifier of the media type, which shall be application/ecss-e-tm-10-25+json.
  • dataModelVersion: version identifier of the ECSS-E-TM-10-25 Annex A Data Model that was used to produce the file.
  • exchangeFileFormatVersion: version identifier of the ECSS-E-TM-10-25 Annex C JSON Exchange File Format (i.e. this specification) that was used to produce the file.
  • creatorOrganization: a JSON object representing the organization that created the archive, with the following properties name, organizationalUnit, locality, iid.
    • name: the common name of the creator organization.
    • organizationalUnit: the name or abbreviation of a unit inside the creator organization (optional).
    • locality: a description of the geographic location of the creator organization (optional).
    • iid: a UUID v4 serialised string that should match the iid of the corresponding Organization object inside the SiteDirectory File (optional).
  • copyright: textual copyright information (optional).
  • creatorPerson: identification of the person responsible for the archive.
    • givenName: given name of the person (optional).
    • surname: surname of the person.
    • email: e-mail address of the person responsible for the archive (optional).
    • iid: a UUID v4 serialised string that should match the iid of the corresponding Person object inside the SiteDirectory File (optional).
  • createdOn: created-on timestamp.
    • utc: a timestamp in ISO 8601 UTC format.
    • local: a timestamp in ISO 8601 local timezone format (optional).
  • lastModifiedOn: last-modified timestamp in ISO 8601 UTC format (optional).
    • utc: a timestamp in ISO 8601 UTC format.
    • local: a timestamp in ISO 8601 local timezone format (optional).
  • remark: free format textual remark (optional).
  • extensions: freely usable JSON object to capture any extensions used by an implementation (optional). It can be used to describe corresponding content in the Extensions Folder of the archive.
• Optional properties shall be assigned the value null when not used.
• For human readability the JSON object should be formatted as indented pretty print

The following snippet provides an example of the content of the header file

{
    "mediaType": "application/ecss-e-tm-10-25+json",
    "dataModelVersion": "2.5.0",
    "exchangeFileFormatVersion": "1.0.0",
    "creatorOrganization": {
        "iid": null,
        "name": "Starion Group",
        "unit": "SEMT",
        "site": "Leiden, The Netherlands"
    },
    "creatorPerson": {
        "iid": "26b47595-371e-45f4-aea5-3dbb7448852f",
        "givenName": "John",
        "surname": "Doe",
        "email": "[email protected]"
    },
    "copyright": "Copyright 2018 © Starion Group S.A.. The information contained in this file may only be used in the context of work on Programme Z by members of the Programme Z development team.",
    "createdOn": {
        "local": "2018-08-20T12:00:00+02",
        "utc": "2015-08-27T10:10:00Z"
    },
    "lastModifiedOn": null,
    "remark": "This is an example ECSS-E-TM-10-25 exchange file created by the CDP4-COMET",
    "extensions": null
}

• The archive contains one SiteDirectory File which is named "SiteDirectory.json".
• The content of the SiteDirectory File is a JSON array of JSON objects, each JSON object is a serialized of an ECSS-E-TM-10-25 object.
• The ECSS-E-TM-10-25 objects contained in the SiteDirectory File comprise the complete composite structure (i.e. the directly and indirectly contained object graph) of an ECSS-E-TM-10-25 SiteDirectory object, with the exception of the composite structures of SiteReferenceDataLibrary and ModelReferenceDataLibrary objects.
• The SiteReferenceDataLibrary and ModelReferenceDataLibrary objects themselves are included in the SiteDirectory File, while their contained object graphs are stored in the SiteReferenceDataLibrary and ModelReferenceDataLibrary Files respectively, as specified below.

• The archive contains one SiteReferenceDataLibrary File for each SiteReferenceDataLibrary object in the SiteDirectory File.
• The SiteReferenceDataLibrary Files are located in a folder in the archive named "SiteReferenceDataLibraries".
• The name of a SiteReferenceDataLibrary File is ".json", where is the iid of the corresponding SiteReferenceDataLibrary object, a UUID v4 serialised string.
• Each SiteReferenceDataLibrary File contains a JSON array of JSON objects that is the serialisation of the complete contained composite structure of the corresponding ECSS-E-TM-10-25 SiteReferenceDataLibrary object, excluding the SiteReferenceDataLibrary object itself. The SiteReferenceDataLibrary object resides in the SiteDirectory File.

• The archive contains one ModelReferenceDataLibrary File for each ModelReferenceDataLibrary object in the SiteDirectory File.
• The ModelReferenceDataLibrary Files are located in a folder in the archive named "ModelReferenceDataLibraries".
• The name of a ModelReferenceDataLibrary File is ".json", where is the iid of the corresponding ModelReferenceDataLibrary object, a UUID serialised string.
• Each ModelReferenceDataLibrary File contains a JSON array of JSON objects that is the serialisation of the complete contained composite structure of the corresponding ECSS-E-TM-10-25 ModelReferenceDataLibrary object, excluding the ModelReferenceDataLibrary object itself. The ModelReferenceDataLibrary object resides in the SiteDirectory File.

• The EngineeringModel Files are located in a folder in the archive named "EngineeringModels".
• The "EngineeringModels" folder contains one EngineeringModel Folder for each EngineeringModelSetup object in the SiteDirectory File. The name of each folder is the serialized iid of the corresponding EngineeringModel object, which is the same as the serialized engineeringModelIid property of the corresponding EngineeringModelSetup object.
• Each EngineeringModel Folder contains the following:
  • One EngineeringModel File named ".json", where is the serialized iid of the corresponding EngineeringModel object. This file contains a JSON array of JSON objects that is the serialisation of the corresponding EngineeringModel object and its contained objects, excluding any Iteration object and objects it contains.
    • An Iterations Folder named "Iterations".
    • A FileRevisions Folder named "FileRevisions".
• The Iterations Folder contains one or more Iteration Files where each Iteration File is named ".json", where is the serialized iid of the corresponding Iteration object_ and contains a JSON array of JSON objects that is the serialisation of the the corresponding Iteration object and its contained objects.
• Typically at least the latest Iteration object will always be included, and optionally older Iteration objects.
• The FileRevisions Folder contains zero or more files, where each file is named by the SHA1 contentHash of the corresponding FileRevision object and the content is its byte stream. In case multiple FileRevision objects have the same SHA1 contentHash, the content is stored only once.

• The archive may contain a folder named "Extensions".
• The Extensions Folder may contain any file or folder content that is used by an implementation in addition to the standardised content as specified in the rest of this file format.
• It is recommended that the content of Extensions Folder conforms to similar JSON formats as the rest of this file format.