The system overview of the Human Navigation is described below.

System Configuration

The Windows computer runs the Human Navigation program. The Human Navigation program has been created in Unity. The Ubuntu computer runs the rosbridge server, the SIGVerse rosbridge server, and the robot controller created by the competition participants.

The Human Navigation program and robot controller communicate through the basic rosbridge server, but communication with a large amount of data (sensor data, etc.) is transmitted through the SIGVerse rosbridge server.

Flow

The flow of the competitive challenge is outlined below.

1. Team member launches the ROSBridge server, SIGVerse ROSBridge server, and robot controller.
2. The operator (Competition committee) launches human_navi.exe
3. The test subject puts on the Oculus Quest 2.
4. The operator presses the session start button.
5. Load the environment for the competitive challenge to initialize the position and direction of the avatar.
6. Moderator sends the “Are_you_ready?” message to the robot controller.
7. The robot controller sends the “I_am_ready” message to the moderator.
8. The moderator sends the TaskInfo to the robot. The TaskInfo includes the target object and destination.
9. The robot controller generates and sends the guidance_message vocal statement to the robot. The robot controller can send guidance_message in an arbitrary timing.
10. The test subject follows the vocal statement to take the target object and to place it on the designated location (destination). The test subject can request a guidance message by pressing the X button. After the test subject requested a new guidance message, the “Guidance_request” message is sent to the robot controller.
    • If the target object is grasped, points will be awarded.
    • If the wrong object is grasped, a penalty will be given.
    • If the target object is placed in/on the destination, points will be awarded.
    • If the number of instruction exceeds 15 times, a penalty will be given.
11. Each session is finished by events below.
    • The target object is placed in/on the destination: The Moderator sends the “Task_succeeded” message to the robot controller.
    • "Give_up" is sent from the robot controller: The robot can send "Give_up" message if it is impossible to achieve the task. In that case, the session is aborted and the Moderator sends the “Task_failed” message to the robot controller.
    • Time is up: The Moderator sends the “Task_failed” message to the robot controller.
12. Session ends.
    • If no sessions are left, the moderator sends “Mission_complete” message to the robot controller.
    • If any sessions are left:
      1. The test subject moves to the next team area and puts on the Oculus Quest 2.
      2. The operator presses the Go_to_next_session button. The Moderator sends “Go_to_next_session” message to the robot controller.
      3. Return to 5.