Contributing to Test Cases - SolidLabResearch/ODRL-Test-Suite GitHub Wiki

Folder Structure

Test cases must be contributed to the data folder which has the following structure:

policy - contains ODRL policies
requests - contains request documents
sotw - contains state of the world (sotw) documents
test_cases - contains test cases

Requirements

Each test case contains at least a policy document, a sotw document and a test_case document.
Optionally a request document can be provided.
For each provided document:
- Write Turtle documents.
- Do not use blank nodes.
- Blank nodes should be translated into URI-s (e.g. UUID URIs as in <urn:uuid:4129123f-d8a8-481e-87fc-aba6dda5b6a5>).
- This transformation of blank nodes is also called "grounding" the graph.

Grounding

Grounding is the process to convert blank nodes into URIs (e.g. UUID URIs).

To ease the grounding of a Turtle document the odrl-test-utils package can be installed:

npm install odrl-test-utils -g

To show the grounded version of a path/to/file.ttl use:

odrl-test-utils ground path/to/file.ttl

To overwrite the file with the grounded version use:

odrl-test-utils ground -x path/to/file.ttl

This grounding program will also convert <../path/to/file> references to the main subject UUID URIs in those files.

Requirements for Test Case Files

In the examples below we assume that a grounder is used to transform the blank nodes in the examples.

Requirements for the Policy File

Each policy file should contain one odrl:Set instance.
This instance should contain an odrl:description property that explains the policy in plain language.

Example policy:

@prefix odrl: <http://www.w3.org/ns/odrl/2/>.
@prefix ex: <http://example.org/>.

<> a odrl:Set;
    odrl:description "Alice is allowed to read resouce X";
    odrl:permission [
       a odrl:Permission;
       odrl:assignee ex:alice;
       odrl:action odrl:read;
       odrl:target ex:resourceX
    ].

Requirements for the Request File

Each request file should contain one odrl:Request instance.
This instance should contain a dct:description property that explains the request in plain language.
This instance typically requests the permission of assignee to do an action on a particular target.

Example request:

@prefix odrl: <http://www.w3.org/ns/odrl/2/>.
@prefix ex: <http://example.org/>.

@prefix dct: <http://purl.org/dc/terms/>.

<> a odrl:Request;
    dct:description "<changeme> description";
    odrl:permission [
       a odrl:Permission;
       odrl:assignee ex:alice;
       odrl:action odrl:read;
       odrl:target ex:resourceX
    ].

Requirements for the Sotw File

Each request file should contain one ex:Sotw instance.
This instance should include a ex:includes property with the subjects that should be part of the sotw.
At least a temp:currentTime subject should be provided which contains as dct:issued a timestamp.

Example minimal sotw:

@prefix ex: <http://example.org/>.
@prefix temp: <http://example.com/request/>.
@prefix dct: <http://purl.org/dc/terms/>.
@prefix xsd: <http://www.w3.org/2001/XMLSchema#>.

<> a ex:Sotw;
    ex:includes temp:currentTime.

temp:currentTime dct:issued "2024-02-12T05:20:10.999Z"^^xsd:dateTime.

Example sotw with a some extra triples:

@prefix ex: <http://example.org/>.
@prefix temp: <http://example.com/request/>.
@prefix dct: <http://purl.org/dc/terms/>.
@prefix xsd: <http://www.w3.org/2001/XMLSchema#>.
@prefix foaf: <http://xmlns.com/foaf/0.1/>.

<> a ex:Sotw;
    ex:includes temp:currentTime, ex:alice.

temp:currentTime dct:issued "2024-02-12T05:20:10.999Z"^^xsd:dateTime.

ex:alice a foaf:Person ;
   foaf:age 16.

Optionally a sotw can include information about fulfillment status of duties.
- This information needs to be provided using an instance of a PolicyReport.

Example sotw with a fulfilled duty:

@prefix ex: <http://example.org/>.
@prefix temp: <http://example.com/request/>.
@prefix dct: <http://purl.org/dc/terms/>.
@prefix xsd: <http://www.w3.org/2001/XMLSchema#>.
@prefix foaf: <http://xmlns.com/foaf/0.1/>.
@prefix report: <https://w3id.org/force/compliance-report#>.

<> a ex:Sotw;
    ex:includes temp:currentTime, _:report.

temp:currentTime dct:issued "2024-02-12T05:20:10.999Z"^^xsd:dateTime.

_:report a report:PolicyReport;
    report:policy <urn:uuid:5aa7f98c-65e0-4ff2-9846-40203203a58a>;
    report:ruleReport[
       a report:DutyReport;
       report:attemptState report:Attempted;
       report:rule <urn:uuid:a0b12cb7-d3a1-4953-86da-f59a597615d2>;
       report:performanceState report:Performed;
       report:deonticState report:Fulfilled
    ].

In this example:

report:policy has as object the grounded identifier of the policy file odrl:Set instance;
report:rule has as object the grounded identifier of the policy odrl:Duty instance.

Example sotw with a violated duty:

@prefix ex: <http://example.org/>.
@prefix temp: <http://example.com/request/>.
@prefix dct: <http://purl.org/dc/terms/>.
@prefix xsd: <http://www.w3.org/2001/XMLSchema#>.
@prefix foaf: <http://xmlns.com/foaf/0.1/>.
@prefix report: <https://w3id.org/force/compliance-report#>.

<> a ex:Sotw;
    ex:includes temp:currentTime, _:report.

temp:currentTime dct:issued "2024-02-12T05:20:10.999Z"^^xsd:dateTime.

_:report a report:PolicyReport;
    report:policy <urn:uuid:5aa7f98c-65e0-4ff2-9846-40203203a58a>;
    report:ruleReport[
       a report:DutyReport;
       report:attemptState report:Attempted;
       report:rule <urn:uuid:a0b12cb7-d3a1-4953-86da-f59a597615d2>;
       report:performanceState report:Unperformed;
       report:deonticState report:Violated.
    ].

Example sotw with an unset duty:

@prefix ex: <http://example.org/>.
@prefix temp: <http://example.com/request/>.
@prefix dct: <http://purl.org/dc/terms/>.
@prefix xsd: <http://www.w3.org/2001/XMLSchema#>.
@prefix foaf: <http://xmlns.com/foaf/0.1/>.
@prefix report: <https://w3id.org/force/compliance-report#>.

<> a ex:Sotw;
    ex:includes temp:currentTime, _:report.

temp:currentTime dct:issued "2024-02-12T05:20:10.999Z"^^xsd:dateTime.

_:report a report:PolicyReport;
    report:policy <urn:uuid:5aa7f98c-65e0-4ff2-9846-40203203a58a>;
    report:ruleReport[
       a report:DutyReport;
       report:attemptState report:Attempted;
       report:rule <urn:uuid:a0b12cb7-d3a1-4953-86da-f59a597615d2>;
       report:performanceState report:Unknown;
       report:deonticState report:NonSet.
    ].

Requirements for the Test Case File

Each test case file should contain one ex:TestCase instance.
This instance should contain an dct:title property that explains the test case in plain language.
This instance should have a ex:policy with the grounded identifier of the odrl:Set instance.
This instance should have a ex:sotw with the grounded identifier of the ex:Sotw instance.
Optionally a ex:request with the grounded identifier of the odrl:Request instance.

Example test case:

@prefix odrl: <http://www.w3.org/ns/odrl/2/>.
@prefix ex: <http://example.org/>.
@prefix dct: <http://purl.org/dc/terms/>.
@prefix report: <https://w3id.org/force/compliance-report#>.

<> a ex:TestCase;
    dct:title "Any request results into yes (Alice Request).";
    ex:policy <urn:uuid:95084b56-20fb-48e2-92bf-268aefd6bfd9>;
    ex:request <urn:uuid:1d765e78-181f-48e9-b5d4-63fa14105435>;
    ex:sotw <urn:uuid:4a27bae7-32ae-4693-9473-63100190acc4>;
    ex:expectedReport [
        a report:PolicyReport;
        report:policy <urn:uuid:95084b56-20fb-48e2-92bf-268aefd6bfd9>;
        report:policyRequest <urn:uuid:1d765e78-181f-48e9-b5d4-63fa14105435>;
        report:ruleReport [ 
           a report:PermissionReport;
           report:attemptState report:Attempted;
           report:rule <urn:uuid:9d4de683-5c29-44de-9cf5-29bb9da0dc26>;
           report:ruleRequest <urn:uuid:f0e623a2-5db7-4597-aca2-82174eb572b9>;
           report:activationState report:Active
        ]
    ].

Where:

report:policy is a copy of ex:policy;
report:policyRequest is a copy of ex:request;
report:rule is the grounded identifier of a odrl:Permission or odrl:Prohibition in the policy file;
report:ruleRequest is the grounded identifier of a odrl:Permission in the request file.

Optionally a test case can contain one or more instances of PremiseReport-s:

ActionReport
ConstraintReport
PartyReport
TargetReport

These reports should be provided as values to the report:premiseReport property.

Example test case with an action report:

@prefix odrl: <http://www.w3.org/ns/odrl/2/>.
@prefix ex: <http://example.org/>.
@prefix dct: <http://purl.org/dc/terms/>.
@prefix report: <https://w3id.org/force/compliance-report#>.

<> a ex:TestCase;
    dct:title "Any request results into yes (Alice Request).";
    ex:policy <urn:uuid:95084b56-20fb-48e2-92bf-268aefd6bfd9>;
    ex:request <urn:uuid:1d765e78-181f-48e9-b5d4-63fa14105435>;
    ex:sotw <urn:uuid:4a27bae7-32ae-4693-9473-63100190acc4>;
    ex:expectedReport [
        a report:PolicyReport;
        report:policy <urn:uuid:95084b56-20fb-48e2-92bf-268aefd6bfd9>;
        report:policyRequest <urn:uuid:1d765e78-181f-48e9-b5d4-63fa14105435>;
        report:ruleReport [ 
           a report:PermissionReport;
           report:attemptState report:Attempted;
           report:rule <urn:uuid:9d4de683-5c29-44de-9cf5-29bb9da0dc26>;
           report:ruleRequest <urn:uuid:f0e623a2-5db7-4597-aca2-82174eb572b9>;
           report:premiseReport [
               a report:ActionReport;
               report:satisfactionState report:Satisfied.
           ];
           report:activationState report:Active
        ]
    ].

Pattern for an ActionReport

[
  a report:ActionReport;
  report:satisfactionState report:Satisfied
]

Pattern for a ConstraintReport

[
    a report:ConstraintReport;
    report:constraint <urn:uuid:f2827c25-a171-4ec6-ab2e-3388e6b76b69>;
    report:constraintLeftOperand "2024-02-12T11:20:10.999Z"^^xsd:dateTime;
    report:constraintOperator odrl:lt;
    report:constraintRightOperand "2024-06-24T17:00:00.000Z"^^xsd:dateTime;
    report:satisfactionState report:Satisfied;
]

where:

report:constraint contains the grounded identifier of the constraint from the policy file;
report:constraintLeftOperand, report:constraintRightOperand can be optional when no object values are available for the test;
report:satisfactionState has value report:Satisfied or report:Unsatisfied;
Optionally deeper nested report:premiseReport properties are available when the report:constraintOperator is boolean.

Example with a deep nested PremiseReport

[
    a report:ConstraintReport;
    report:constraint <urn:uuid:9357eff0-74ad-4ef2-b7cd-b711773f361b>;
    report:constraintLogicalOperand odrl:and;
    report:premiseReport [
        a report:ConstraintReport;
        report:constraint <urn:uuid:f2827c25-a171-4ec6-ab2e-3388e6b76b69>;
        report:constraintLeftOperand "2024-02-12T11:20:10.999Z"^^xsd:dateTime;
        report:constraintOperator odrl:lt;
        report:constraintRightOperand "2024-06-24T17:00:00.000Z"^^xsd:dateTime;
        report:satisfactionState report:Satisfied;
    ],
    [
        a report:ConstraintReport;
        report:constraint <urn:uuid:f2827c25-a171-4ec6-ab2e-3388e6b76b69>;
        report:constraintLeftOperand ex:age
        report:constraintOperator odrl:gt;
        report:constraintRightOperand 16;
        report:satisfactionState report:Satisfied;
    ]
    report:satisfactionState report:Satisfied
]

Pattern for a PartyReport

[
  a report:PartyReport;
  report:satisfactionState report:Satisfied
]

Pattern for a TargetReport

[
  a report:TargetReport;
  report:satisfactionState report:Satisfied
]

Test the Test Case

Run the demo/test-case.ts command with the grounded identifier of the test case to test the setup:

npx ts-node demo/test-case.ts <grounded-identifier-of-test-case>

Helper Tools

The odrl-test-utils command can be used the generate sample files for policy, request, sotw and test case files:

# Create a sample policy
odrl_test_utils sample policy

# Create a sample request
odrl_test_request sample request

# Create a sample state of the world (sotw)
odrl_test_utils sample sotw

# Create a sample test case
odrl_test_utils sample testcase

# Create a sample test case for a prohibition
odlr_test_utils sample --type prohibition testcase

# Create a sample test case with an action, constraint, party and target report
odrl_test_utils sample --action --constraint --party --target testcase