To connect ROS2 to Unity, you will need to do some setup both in Unity and for ROS2. You should reference Unity's setup instructions in addition to this guide if you need more detail.

1. Make sure you have ROS2 installed.

• You can install ROS2 for Ubuntu or Windows 10.

2. Create a workspace.

• In ROS, a workspace is just a directory that holds all of the different packages for your project. You can create this with mkdir unity_connector_ws.

3. Add the ROS TCP Endpoint package to your workspace.

• In unity_connector_ws, create a src folder.

• Clone the ROS TCP Endpoint repository to the src folder. Make sure you clone the main-ros2 branch by running the following command:

  git clone -b main-ros2 https://github.com/Unity-Technologies/ROS-TCP-Endpoint.git

4. Build the package by running colcon build in the unity_connector_ws folder (you must be out of the src folder).
5. Source the installation with source install/setup.bash(Linux) or call install/setup.bash(Windows)

Note: If you are running wsl on Windows use source install/setup.bash.

Once you have the package installed, run the TCP endpoint with: ros2 run ros_tcp_endpoint default_server_endpoint --ros-args -p ROS_IP:=<your IP address> Note: Remove both ANGLED brackets and type in your IP Address

• ROS_IP should be set to the IP of your machine. You can find this with ifconfig(Linux) or ipconfig(Windows). Or, if you are using WSL and this is also if you are using WSL on Windows, wsl hostname -I in a Powershell Terminal to find the IP ADDRESS of YOUR MACHINE (if you are already in wsl use the command hostname -I).
• Make sure you have run source install/setup.bash in the workspace folder (in this case, unity_connector_ws).
• If you are using Windows 10, call install\setup.bat instead

If you expect to connect to the Create3 hardware, you will likely run into a ROS2 QoS error, due to incompatible default QoS profiles between Unity's ROS2 TCP Endpoint and the ROS2 nodes on the Create3. To fix this issue, add the line qos_profile.reliability = QoSReliabilityPolicy.BEST_EFFORT to line 48 of ROS-TCP-Endpoint\ros_tcp_endpoint\subscriber.p in the src directory of your ROS workspace.

• Rebuild this package with colcon build and source install/setup.bash

1. Make sure you have the Unity Editor installed with a version greater than 2020.

2. Create a new Unity project and Open Unity Editor

3. Open Window -> Package Manager

4. Click the "+" button in the upper left corner and choose Add package from git URL....

   

5. Enter https://github.com/Unity-Technologies/ROS-TCP-Connector.git?path=/com.unity.robotics.ros-tcp-connector

6. Click Add

   • If upon clicking 'Add' and nothing happens in response (no notification that it is being installed), go to the https://github.com/Unity-Technologies/ROS-TCP-Connector download the whole repo, unzip, and store it where you can find it. Next in Unity click add from disk and navigate to ROS-TCP-Connector-main> com.unity.robotics.ros.tcp-connector> package.json

You should now have the Unity Package installed under Packages - Other, and you should see a Robotics tab at the top of the editor.

Before we can connect Unity and ROS, we'll need to create some scripts in Unity. These will include C# classes generated by the the Unity TCP Connector package and a subscriber script that we will write.

ROS uses "topics" to share data. You can find more information in the ROS documentation. For now, all we need to know is that we "subscribe" to a topic to get data from it, and we "publish" to a topic to push data to it.

To interpret that data, ROS uses "messages". A message is like a data type declaration. Topics will share messages of a specific type, which you can check with ros2 topic info <topic_name> -v.

The Unity Package provides a utility to generate C# classes based on these message types to streamline reading and writing within Unity's C# scripts. We are going to need to provide specific msg files to the Unity Package.

• You can download the ROS default message declarations here.
• You can download the iRobot Create3's message declarations here.

Download the zip files from these repositories and extract them wherever you like, just keep track of where they are stored.

Now, we will pass this message information to the Unity Package to create the C# classes.

1. Under the Robotics tab at the top of the Unity Editor, open ROS Settings and make sure the Protocol is set to ROS2.
2. Under the Robotics tab, click on Generate ROS Messages...
3. In the ROS message path field, you will need to either paste in or browse to a msg folder (or, a folder containing a msg folder).

• The ROS common interfaces as a lot of subgroups. You only need to add the folders that contain messages you need. You can check this with the ros2 topic info <topic_name> -v command.

4. Click on the build <message_number> msgs button to generate the C# scripts.

For more details, including screenshots, refer to the final section (Install Unity Robotics Demo) of this guide.

You should now have C# scripts in /Assets/RosMessages in your Unity Project.

To subscribe to a topic, that is, to read data from ROS into Unity, we will need to write a C# script in Unity to do the subscribing. In your Unity Assets, create a new C# script named ""ROSSubscriber". A basic subscriber script looks like this:

using UnityEngine;
using Unity.Robotics.ROSTCPConnector;
using RosMessageTypes.Std;

public class ROSSubscriber : MonoBehaviour
{
    public string TopicName;

    void Start()
    {
        ROSConnection.GetOrCreateInstance().Subscribe<StringMsg>(TopicName, TopicFunction);
    }

    void TopicFunction(StringMsg MyMessage)
    {
        Debug.Log(MyMessage);
    }
}

Copy paste this code into your ROSSubscriber.cs script.

For this example, I'm going to use ROS's demo nodes. Your ROS2 installation should come with some demo nodes. Specifically, we want the talker demo node. Open a new window in 'wsl' and run this node with ros2 run demo_nodes_cpp talker. This will run a node that publishes to the topic /chatter. You can see the details of /chatter with ros2 topic info /chatter -v. Notice that the "Topic Type" is std_msgs/msg/String. This means the message declaration for this topic will be in the ROS2 common interfaces repo, in the /std_msg/msg folder in the String.msg file.

This means that, if you have built the msg files in /std_msg in Unity, there will now be C# scripts in /Assets/RosMessages/Std/msg. One of these should be StringMsg.cs.

On the third line of the above basic script, you will see the line using RosMessageTypes.Std; This is what allows us to access the C# classes for the message types we want to use, and built using the ROS Message Browser. To add other folders with the C# scripts for other messages, you can add another using statement, in the format using RosMessageTypes.<name_of_folder>;.

Now, we need to set up a Subscriber GameObject:

1. Create an empty GameObject named "Subscriber".
2. Click on "Subscriber". A 'Inspector' tab should open on the right, with a Transform property being the only thing listed. Navigate to /Assets/ROSSubscriber and drag and drop the file into the empty space below the Transform property, thereby adding the ROSSubscriber.cs script to this GameObject.
3. In the Topic Name: field of your "ROS Subscriber" component of your "Subscriber" object, write chatter. This is the name of the topic we wish to subscribe to.

You should now have a Subscriber object with the following properties:

In order to connect our Unity subscriber to the ROS2, we will need the following things:

1. An ROS2 node publishing to a topic. In this case, we will have the talker node publishing to chatter. Start this with ros2 run demo_nodes_cpp talker if it is not already running.
2. An ROS2 TCP Endpoint. Start this with ros2 run ros_tcp_endpoint default_server_endpoint --ros-args -p ROS_IP:=<your IP address> if it is not already running.
3. Configure the Unity TCP Connector with the correct IP address. Under Robotics -> ROS Settings set the ROS IP Address field to the same IP address that you have passed to the ROS TCP endpoint.

If you have a talker node publishing to chatter, a ROS2 TCP Endpoint running, and Unity's TCP Connector configured with the TCP endpoint IP address, you can hit the play button in the Unity Editor. If you open the Unity Console, you should see the messages being printed by the talker node appearing in the Unity Console.