OSC

OSC is a Greta module for communication management between applications based on OSC protcol.

OSC_Sender class which will take care of sending and receiving messages to and from scripts in Unity. This class is also a graphical interface.

This class contains 3 attributes to describe the state of the connection:

• a boolean attribute linked to a checkbox to know whether or not to use OSC,
• an attribute of type OSCPortOut
• an int attribute for the definition of the connection port to use.

Concerning the methods we have a method to start sending messages “startOSCout”, a method to stop sending messages “stopOSCout”, a method for sending a sendOSC message that supports a String (the recipient of the message) and a Map<String,Double> structure for sending information in the message, where the String attribute is the address to which the message should be sent. For example, send to recipient X at the address "unity/fruit/name" the object Y

We have another checkbox jCheckBox1 which will take care of starting the reception of messages. Once checked, a thread will start listening for incoming messages. The target messages are those that contain information from Unity about the objects and agents in the scene. These items are then stored in a list and displayed in the GUI.

With this approach, one can select an item or agent from the list, send it with OSC and then a Unity GRETA agent will look in the direction of the selected object/agent

Scripts in GRETA_UNITY :

• Event Receiver: this Script(component) will allow you to receive messages from GRETA on a specific port at a specific address. There are two modules in Unity:
  • module to receive messages in String form (the name of the object or agent to look at)
  • module that will handle a vector3(x,y,z) to look at a point in space. Each Character must have its own eventReceiver
• OSCNE: it is a Script associated with the Scene object which is in charge of sending the list of observable objects/agents in the Scene. To find these objects we browse the set of "children" of the Scene object and we store in a list the objects whose name starts with # (for example #Paul, #tomato)

Mechanism for gaze_to_target

Thanks to the package in Unity AnimationRigging we can generate constraints on objects. For example, we can impose the constraint on the agent's head to look at a specific position; the same is true for the rest of the body.

Different elements are used for this:

• RigBuilder: module associated with Character, it will allow us to manage the rigs
• Rigs1: an object that contains rigs, one for each part of the body on which you want to impose a constraint
• MovingObject: an invisible object that will move into the position of the object/agent you are looking at
• ChestAim,HeadAim ….: These are the constraints on the torso, on the head etc. These objects all have a module of type Multi-Aim Constraint which will have, as ConstraintObject:
  • the bone subject to the constraint,
  • a sourceObject which will be the MovingObject to be followed,
  • the axes of rotation and
  • rotation limits

In the end, we get the name of an object/agent to look at, the rigs will all have the MovingObject as the object to look at, the MovingObject will move to the position of the object/agent to look at and the torso/head/etc will rotate to look at that object