Action Definitions - aantakli/AJAN-service GitHub Wiki

In AJAN, MMUs are executed using the action nodes described in chapter AJAN Actions. For this purpose, two Action Definitions were created as PluginActions and presented below.

To be able to execute a MMU in MOSIM, so-called MInstructions are required. To be able to execute an MInstruction three data are needed in general: a name and a generated ID of this instruction; and to know which MMU is to be executed, the motion type or the ID of the MMU to be used.

But in most cases a MMU needs more context information to create an animation. For this a MInstruction has two more fields or lists. The MInstruction.Properties list contains string-based key-value pairs with required or optional properties. The second list MInstruction.Constraints contains MConstraints. Below is an example shown of the of Locomotion/Walk MMU with its listed properties or parameters and how to create and execute an MInstruction for that MMU in C#:

MInstruction walkInstruction = new MInstruction(MInstructionFactory.GenerateID(), "Walk", "Locomotion/Walk")
{
     Properties = PropertiesCreator.Create("TargetID", UnitySceneAccess.Instance.GetSceneObjectByName("WalkTarget").ID)
};

this.CoSimulator.AssignInstruction(walkInstruction, currentState);

As can be seen, the Locomotion/Walk MMU needs a defined TargetID or TargetName as property to know to which target the avatar should walk through this MMU. In the shown C# example a MSceneObject with the name "WalkTarget" is read and its ID is passed to the MMU.

In order for AJAN to create and execute an MInstruction with said properties and constraints, the required information must be read out via an SBT Action node as Action Input and is used by an Action Definition to send the CoSimulator the Thrift based MInstruction instruction.

Implementation: https://github.com/aantakli/AJAN-service/blob/master/pluginsystem/plugins/MOSIMPlugin/src/main/java/de/dfki/asr/ajan/pluginsystem/mosimplugin/extensions/instructions/AsyncMMUInstruction.java

To execute a MMU asynchronously in AJAN this Action Definition is needed. Here an MMU is executed and only considered finished when AJAN receives an "end" or an "initError" event from the CoSimulator. MMUs (such as the Object/Carry MMU) can have additional events. Which event should be obeyed to consider an action as finished is defined by the Action Input.

To execute an AJAN action, whether ServiceAction or PluginAction based, with an Action Definition, an Action Input with said information is required. What information is needed is specified with a Consumable. In the case of this action definition, the following Consumable is used:

	actn:consumes [
		a actn:Consumable ;
		actn:sparql """
			PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>
			PREFIX mosim: <http://www.dfki.de/mosim-ns#>
			PREFIX actn: <http://www.ajan.de/actn#>
			ASK
			WHERE {
				?instruction rdf:type mosim:MInstruction .
				?instruction mosim:mmu ?mmu .
				OPTIONAL {
					?instruction mosim:actionName ?actionName .
					?instruction mosim:finalEvent ?event .
					?instruction mosim:mmuProperty ?property .
					?instruction mosim:constraint ?constraint .
					?instruction mosim:startCondition ?startCond .
					?instruction mosim:endCondition ?endCond .
				}
				?cosim rdf:type mosim:CoSimulator .
				?cosim mosim:host ?host .
				?cosim mosim:port ?port .
			}"""^^xsd:string ;
	] ;

As shown in the Consumable, basically two pieces of information are needed to execute an MMU with AJAN: (1) a instruction resource ?instruction with a the Motion Type mosim:mmu (xsd:sting); (2) information about the CoSimulator mosim:CoSimulator with its host address mosim:host (<xsd:sting>) and port mosim:port (<xsd:integer>) to be used. Optionally, MInstruction properties mosim:mmuProperty (<URI> whith whole resource description), constraints mosim:constraint (<URI> whith whole resource description), start mosim:startCondition (<xsd:sting>) and end mosim:endCondition (<xsd:sting>) conditions, and an action name mosim:actionName (<xsd:sting>) can be specified. If and which of these data must be defined depends on the MMU to be used.

Basically, the needed CoSimulator resource (represented in the example with ?cosim) from RDF type mosim:CoSimulator (where mosim: is the @prefix of http://www.ajan.de/mosim-ns#) has the following parameters:

Subjects:

• <CoSimulator> is a RDF Blank Node or a URI representing a CoSimulator Resource

Namspaces:

• @prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>
• @prefix mosim: <http://www.ajan.de/mosim-ns#>

RDF Structure: CoSimulator

RDF Example: CosIMulator

@prefix mosim: <http://www.dfki.de/mosim-ns#> .
@prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> .

_:cosim rdf:type mosim:CoSimulator .
_:cosim mosim:host "http://localhost" .
_:cosim mosim:port 9011 .

Basically, the needed MInstruction resource (represented in the example with ?instruction) from RDF type mosim:MInstruction (where mosim: is the @prefix of http://www.ajan.de/mosim-ns#) has the following parameters:

Subjects:

• <MInstruction> is a RDF Blank Node or URI representing a MInstruction Resource
• <Property> is a RDF Blank Node representing a Property Resource
• <Constraint> is a RDF Blank Node representing a MConstraint Resource

Namspaces:

• @prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>
• @prefix mosim: <http://www.ajan.de/mosim-ns#>

RDF Structure: MWalkPoint

Optional RDF Predicate: Action Name, AJAN specific and only used, if a instruction needs to be accessed via its "Name" later in the SBT:

Optional RDF Predicate: Final Event, An additional MMU-based event that AJAN listens for, to recognize an AsyncMMUInstruction-based action as completed. This event overwrites the standard CoSimulator based event "end":

Optional RDF Resource: Propery, MInstruction.Properties<string,string>:

Optional RDF Resource: Constraint, MInstruction.Constraints:

Optional RDF Predicate: Start/End Conditions:

RDF Example: Instruction

@prefix mosim: <http://www.dfki.de/mosim-ns#> .
@prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> .

test:Instruction mosim:mmu "Locomotion/Walk" .
test:Instruction mosim:actionName "WalkX" .
test:Instruction mosim:mmuProperty test:TargetProperty .
test:Instruction mosim:constraint test:Constraint .

test:TargetProperty rdf:type mosim:MProperty .
test:TargetProperty mosim:key "TargetID" .
test:TargetProperty mosim:value "12" .

test:Constraint rdf:type mosim:MConstraint .
test:Constraint mosim:object ?object .

	actn:produces [
		a actn:Producible ;
		actn:sparql """
			ASK
			WHERE {
				?s ?p ?o .
			}"""^^xsd:string ;
	] ;

Since MMUs strictly speaking do not have a return value and only emit an event, this PluginAction independently generates an ActionResult and stores it in the agent's knowledge base. For this reason, this Producible only checks whether an RDF record was created. The structure of the output or the ActionResult is explained in the chapter MOSIM Plugin Instruction.

@prefix mosim: <http://www.dfki.de/mosim-ns#> .
@prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> .
@prefix test: <http://test/> .

test:Instruction mosim:mmu "Locomotion/Walk" .
test:Instruction mosim:actionName "WalkX" .
test:Instruction mosim:mmuProperty test:TargetProperty .
	
test:TargetProperty rdf:type mosim:MProperty .
test:TargetProperty mosim:key "TargetID" .
test:TargetProperty mosim:value "12" .

_:cosim rdf:type mosim:CoSimulator .
_:cosim mosim:host "http://localhost" .
_:cosim mosim:port 9011 .

Implementation: https://github.com/aantakli/AJAN-service/blob/master/pluginsystem/plugins/MOSIMPlugin/src/main/java/de/dfki/asr/ajan/pluginsystem/mosimplugin/extensions/instructions/MMUInstruction.java

To execute a MMU synchronously in AJAN this Action Definition is needed. Here an MMU is executed and then considered as finished without waiting for a corresponding event (e.g. "end") from the CoSimulator. But this means that AJAN does not notice if a MMU is not executable. The Pose/Idle could be seen as an example for a synchronous execution of a MMU, where AJAN does not have to wait until this is completed, as it must be explicitly terminated.

Since the not waiting for an event is the only difference between the MMUInstruction presented here and the AsyncMMUInstruction described earlier, this Action Definition will not be discussed further below. The only difference for the Action Input to be defined is that the field or property mosim:finalEvent is not considered for action execution and thus not considered by Consumable shown below.

	actn:consumes [
		a actn:Consumable ;
		actn:sparql """
			PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>
			PREFIX mosim: <http://www.dfki.de/mosim-ns#>
			PREFIX actn: <http://www.ajan.de/actn#>
			ASK
			WHERE {
				?instruction mosim:mmu ?mmu .
				OPTIONAL {
					?instruction mosim:actionName ?actionName .
					?instruction mosim:mmuProperty ?property .
					?instruction mosim:constraint ?constraint .
					?instruction mosim:startCondition ?startCond .
					?instruction mosim:endCondition ?endCond .
				}
				?cosim rdf:type mosim:CoSimulator .
				?cosim mosim:host ?host .
				?cosim mosim:port ?port .
			}"""^^xsd:string ;
	] ;

	actn:produces [
		a actn:Producible ;
		actn:sparql """
			ASK
			WHERE {
				?s ?p ?o .
			}"""^^xsd:string ;
	] ;