The system overview of the Interactive Cleanup is described below.

System Configuration

The system configuration for this competitive challenge is outlined below.

The Windows computer runs the Interactive Cleanup program. The Interactive Cleanup program has been created in Unity.
The Ubuntu computer runs the rosbridge server, the SIGVerse rosbridge server, and the robot controller created by the competition participants.

The Interactive Cleanup program and robot controller communicate through the basic rosbridge server, but communication with a large amount of data (sensor data, etc.) is transmitted through the SIGVerse rosbridge server.

In the Interactive cleanup program, robots move in accordance with the instructions from the robot controller when the human avatar issues the Cleanup command to the robot.
The instructions for the Cleanup command are determined based on the operations of the human avatar and the messages sent by the human avatar.

The robot controller sends ROS messages such as Twist and JointTrajectory to the Interactive Cleanup program to move the robot in the Interactive Cleanup program.

The Interactive Cleanup program distributes JointState, TF, sensor information, and other ROS messages at a regular interval to the Robot Controller.

Flow of the competition

The flow of the competitive challenge is outlined below.

1. Launch the robot controller, the SIGVerse rosbridge, etc on Ubuntu side.
2. Launch the Interactive Cleanup program on Windows side.
3. Initialize the position and direction of the robot and object to grasp.
4. The avatar sends the “Are_you_ready?” message to the robot.
5. The robot sends the “I_am_ready” message to the avatar.
6. The avatar sends the Cleanup command.
   1. The avatar moves to point to the object for Cleanup.
   2. The “Pick_it_up!” message is sent to the robot.
   3. The avatar moves to indicate the location of the object for Cleanup.
   4. The “Clean_up!” message is sent to the robot.
   5. The avatar moves back to initial position.
   6. The robot can request a re-pointing optionally.
      If the robot send the “Point_it_again” message, this step is repeated. (re-pointing)
7. The robot moves to grasp the object for cleanup.
   1. The robot closes the gripper to clasp the object to grasp.
   2. Confirmation of whether the object is correct can be performed optionally.
      The robot will send the “Is_this_correct?” message after grasping the object, and then the avatar responds either “Yes/No”. However, a deduction is taken for each confirmation.
8. The robot sends the "Object_grasped" message to the avatar.
9. The robot moves and releases the object for cleanup.
10. The robot sends the “Task_finished” message to the avatar.
11. The avatar checks the status of Cleanup and assigns points. The task ends.

• If the task ends (successfully or unsuccessfully):
  The avatar sends a “Task_succeeded” (successful task) or “Task_failed” (unsuccessful task) message to the robot to start the next task when participants still have attempts left.
  The avatar sends the “Mission_complete” message to the robot to end the competitive challenge when participants have no attempts left.
• If the time limit has passed:
  The avatar sends the “Task_failed” message to the robot to indicate the task was unsuccessful.
• If users would like to withdraw from the task:
  Users press the Give Up button on the screen to have the avatar send the “Task_failed” message to the robot and indicate the task was unsuccessful.
• Results of Cleanup:
  The Cleanup will be deemed a success if the object for clean-up is placed on the table or disposed of in the garbage box pointed to by the avatar.
• The robot can send “Give_up” message if it is impossible to achieve the task. In that case, the task is aborted and “Task_failed” message is sent then go to the next session.