Neuroevolution Exercise in Roomba Environment

In this assignment, we will be solving a multi-agent coordination problem. That is, given a fixed number of Roomba robots with no prior knowledge of crumbs on the floor and chairs that might block the robots, how could these robots collect as many crumbs as possible?

Part 1: Adding Sensors

Currently, the rtNEAT agents use only the following two sensors:

1. Angle of the closest crumb (between -180 and +180)
2. Fraction of all room crumbs that are in agent's detection radius (1/8 of the room's width)

Add one or more new sensors to this set and run the evolution again. Compare your results with the baseline implementation. Below are some suggestions for new sensors:

1. Distance to the closest crumb
2. X coordinate of the robot's position
3. Y coordinate of the robot's position

Part 2: Communication and Coordination

Start again with the two existing sensors. This time, add sensors allowing Roombas to know where the other Roombas are at each time step. You can also include sensors encoding the number of crumbs collected by the other Roombas. Run evolution again and compare your results with the baseline implementation.

Plotting Fitness

Every time you finish running a particular type of agent and click "exit" at the control panel, the agent's fitness scores are saved in a csv file. You can find the file in OperNERO's root directory under the name roomba-[date].[time].csv. The first column in the csv file records the episode numbers, and the second column records the total number of crumbs collected by all running agents in that episode.

An example csv file looks like this:

1, 9
2, 50
3, 35
4, 40
5, 121
6, 187
7, 224

There is also a utility script called plot_csv.py in OpenNERO's root directory which can plot your saved csv file. The script expects the name of the csv file as its first argument. Here is an example:

python plot_csv.py roomba-2015-12-29.02-06-56.csv

Below is an example plot generated by this script:

Your Report

After doing the experiments, respond to the following questions:

• What kind of sensors, from the point of view of individual agents, did you add? How did it affect the learning process?

• What happens when you give an agent some information on other agents (e.g. positions, number of crumbs collected, etc)? Does this type of communication between multiple agents improve the learning process?

Note that we do not expect you to develop agents that collect all the crumbs within the room.

Module Files

Below is some explanation on each file in the Roomba directory:

• action_script.py – movement commands for Roomba agents.
• agent_handler.py – data structures for agent state, and initialization of agents.
• world_handler.py – places the crumbs.
• world_configure.txt – user-specified information for the positions of crumbs.
• constants.py – environment and agent constants.
• module.py – Roomba environment setup and the sensors. Modify this file to add new types of agents or new types of sensors.
• RTNEATAgent.py – real-time NEAT agent.
• RLAgent.py – tiling tabular reinforcement learning agent.
• client.py – UI controls for the Roomba mod.

References

For more about Roomba Environment, please check out: https://github.com/nnrg/opennero/wiki/RoombaMod

For more about neuroevolution mechanism, please check out: https://github.com/nnrg/opennero/wiki/NeuroEvolution.

For the paper that characterizes rtNEAT information, please check out: http://nn.cs.utexas.edu/?stanley:ieeetec05.

Debugging

If you run into any bugs in your program, you can find the error log file for OpenNERO at one of the following locations:

• Linux or Mac: ~/.opennero/nero_log.txt
• Windows: "AppData\Local\OpenNERO\nero_log.txt" or "Local Settings\Application Data\OpenNERO\nero_log.txt" depending on the version of Windows you have.