BaseBridge

The BaseBridge class provides a high-level foundation for managing communication between Unreal Engine and a reinforcement learning agent running externally (e.g., in Python). It handles connection setup, socket communication, and tick-based update integration.

This class is meant to be subclassed by more specific implementations like SingleEnvBridge and MultiEnvBridge, which provide detailed environment behaviors.

---

Overview

The base bridge sets up a TCP socket using an Unreal-native socket connection class and orchestrates the RL update cycle. It handles message exchange and training/inference switching. It is designed to be extended with domain-specific logic.

On connection, the bridge sends the following handshake:

CONFIG:OBS={ObservationSpaceSize};ACT={ActionSpaceSize}

Subclasses can override BuildHandshake() to customize the format.

---

Usage

Users should extend this class and override UpdateRL, CreateTcpConnection, and other virtual methods to build a working bridge for their specific simulation scenario. This class is a pure C++ UObject and does not rely on scene placement or physics ticking. It does not define or manage environment-specific callbacks—that responsibility lies with subclasses.

---

Initialization and Configuration

bool Connect(const FString& IPAddress, int32 Port, int32 ActionSpaceSize, int32 ObservationSpaceSize)

Initializes internal dimensions and starts listening for a connection. Returns true on success. This function will create a TCP connection instance via CreateTcpConnection() and start listening on the specified address and port.

FString BuildHandshake()

Generates the handshake string sent to the Python agent. Default format is:

CONFIG:OBS={ObservationSpaceSize};ACT={ActionSpaceSize}

Override this if your training script requires additional metadata.

void Disconnect()

Closes the TCP socket and resets the connection object.

UBaseTcpConnection* CreateTcpConnection()

Pure virtual. Subclasses must return a concrete UBaseTcpConnection-derived object to manage low-level socket messaging.

---

Execution Control

void StartTraining()

Enables training mode by setting bIsTraining = true. Inference mode is automatically disabled.

void StartInference()

Enables inference mode by setting bIsInference = true. Training mode is automatically disabled.

void UpdateRL(float DeltaTime)

Virtual method meant to be overridden in subclasses. Called every frame through Tick. Subclasses implement the core training or inference loop here.

---

Model and Action Interface

bool SetInferenceInterface(UInferenceInterface* Interface)

Registers a local model that implements RunInference. This is used in inference mode instead of sending/receiving actions over the network.

FString RunLocalModelInference(const FString& Observation)

Parses the observation string and passes it to the registered inference interface. Returns the resulting action string.

---

TCP Communication

bool SendData(const FString& Data)

Appends \n delimiter to the message and sends it to the connected Python agent. Returns true if successful.

FString ReceiveData()

Reads a message from the TCP connection and returns it as a string. Returns an empty string if disconnected or failed. Expects \n delimiter.

---

Ticking Methods

void Tick(float DeltaTime)

Unreal tick function that delegates to UpdateRL(DeltaTime) each frame.

bool IsTickable() const

Returns true if the bridge is active (training or inference) and socket is valid.

TStatId GetStatId() const

Returns the stat group used to track tick performance.

---