Recognizing Human Pointing Direction with TurtleBot3

In this example, TurtleBot3 recognizes the pointing direction of the human avatar and rotates the Manipulator in that direction.

Human detection is performed using yolo_ros(YOLO).

Prerequisite

Please refer here to install and build required packages.

Startup Procedure

Start the Ubuntu side and then the Windows side.

Ubuntu Side Startup Procedure

Open a new terminal and run the following command:

ros2 launch sigverse_turtlebot3 recognize_pointed_direction_launch.xml

Windows Side Startup Procedure

Start the [Assets/SIGVerse/ExampleScenes/Turtlebot3/Pointing(.unity)] scene with reference to this page.

Run

When you press the left and right arrow keys after focusing on the Unity window, the right hand of the human character moves.
Then press the Space key and TurtleBot3 will rotate the Manipulator in the direction that the human character points.

If you want to finish, stop the Unity side and then the ROS side.

Ubuntu side  (It might differ from the latest version)

Windows side  (It might differ from the latest version)

---

Reference Video  (It might differ from the latest version)