Here is an overview of the project's structure:

images/structure.png

There are two main components that implement the robot controller:

• The controller itself (vpg_controller)
• The libtorch shared object (libvpg_network.so)

All dependencies to libtorch are contained within the shared object. This makes the development of the controller and neural network functionalities separate and makes it easy to verify the behavior and good functioning of the implemented neural networks and related functionalities independently from Webots (see the unit_test/ folder). The controller is dynamically linked to the shared object.

Webots project

The home folders contains several scripts to build and clean the project:

• env.sh: needs to be sourced before using the other scripts.
• build_project.sh: builds both the shared object and the controller.
• clean_project.sh: cleans both the shared object and the controller.

controllers/

There are two controllers:

khepera_test/

This controller implements a simple obstacle avoidance controller based on the demo controller khepera.c provided by Cyberbotics. It makes use for this of the same functions that are used by the trained agent to control the robot defined in libraries/utils/khepera.cpp.

vpg_controller/

This is the project's main controller and arbitrates the training and evaluation of the neural network. The logs and models trained are stored within this folder.

libraries/

It contains two elements:

• networks/vpg_network/: All the code implementing the shared object allowing to generate and train a neural network. This is the only part of the project relying on libtorch.
• utils/: this folder contains more general source code such as an interface to control the robot (khepera.cpp), reward computations (reward.cpp), state measurement (state.cpp) and utility functions to help handle the training (training_handler.cpp).

log_analyzer/

The logs generated as a csv file during training can be plotted at any time during or after training by the provided python script vpg_analyzer.py. This requires python3, with installed packages: numpy, pandas, matplotlib.

unit_test/

A series of simple tests are implemented in vpg_test.cpp to verify the good functioning of the shared object.

worlds/

The world in which the robot is trained in is learned_obstacle_avoidance.wbt and simply consists of a square arena with some obstacles.