ForagerRL_step1 - gama-platform/gama GitHub Wiki
By Killian Trouillet
This is Part 1 of the internal Reinforcement Learning tutorial in GAMA.
| Step | Title | Concepts |
|---|---|---|
| 1 | The Grid World |
grid, global, experiment, display
|
| 2 | The Forager Agent |
species, reflex, aspect, switch/match
|
| 3 | Rewards and Episodes | Reward function, episode management, monitor
|
| 4 | The Q-Table |
map<string, float>, state representation |
| 5 | Q-Learning Algorithm | Epsilon-greedy policy, Bellman equation |
| 6 | Charts, Test Mode and Visualization |
chart, heatmap, training stop, automatic test |
This first step illustrates how to create a grid environment in GAMA. We define a 10×10 grid with round food and rectangular obstacles.
- Definition of global parameters for grid size, food position, and obstacle positions
- Definition of a
world_cellgrid withis_foodandis_obstacleattributes - Display of the grid with colored cells: dark gray for obstacles, white for empty, and a green circle for the food
The global section defines the world parameters. We store the grid size, the food coordinates, and a list of obstacle positions as point values.
global {
int grid_size <- 10;
int food_x <- 9;
int food_y <- 9;
list<point> obstacle_positions <- [{2,2}, {3,2}, {2,3}, {6,4}, {7,4}, {7,5}];
-
int grid_size <- 10: Defines the size of our grid (10 columns × 10 rows). -
list<point>: A list of coordinate pairs. Each{x, y}represents the position of an obstacle.
In the init block, we configure the grid cells by setting their is_food and is_obstacle attributes. The operator grid_at allows us to access a specific cell by its coordinates.
init {
ask world_cell grid_at {food_x, food_y} {
is_food <- true;
}
loop pos over: obstacle_positions {
ask world_cell grid_at pos {
is_obstacle <- true;
}
}
}
}
-
ask world_cell grid_at {x, y} { ... }: Selects the grid cell at coordinates (x,y) and executes the block on it. -
loop pos over: list { ... }: Iterates over each element of the list.
A grid in GAMA is a special type of species where each cell is an agent. We define two boolean attributes (is_food, is_obstacle) and a dynamic color attribute that uses the update facet to reflect the cell state at every simulation step.
grid world_cell width: 10 height: 10 neighbors: 4 {
bool is_food <- false;
bool is_obstacle <- false;
rgb color <- #white update: is_obstacle ? rgb(60,60,60) : #white;
}
-
width: 10 height: 10: Creates a 10×10 grid of cells. -
neighbors: 4: Uses Von Neumann neighborhood (up, down, left, right only — no diagonals). -
update:facet: Used here to color obstacles only. The food will be drawn separately.
The experiment block defines how to run and visualize the simulation.
experiment smart_forager type: gui {
output {
display "Grid World" {
grid world_cell border: #lightgray;
graphics "food" {
ask world_cell where each.is_food {
draw circle(5) color: rgb(50, 180, 50);
}
}
}
}
}
-
type: gui: A graphical experiment with a user interface. -
grid ... border: #lightgray: Displays the grid cells. -
graphics "food" { ... }: A graphics layer to draw objects manually. We ask every food cell to draw a circle at its location.
/**
* Name: SmartForager - Step 1: The Grid World
* Author: Killian Trouillet
* Description: This first step creates a simple 10x10 grid environment
* with food (green cell) and obstacles (dark gray cells).
* Tags: reinforcement-learning, grid, tutorial
*/
model SmartForager
global {
int grid_size <- 10;
int food_x <- 9;
int food_y <- 9;
list<point> obstacle_positions <- [{2,2}, {3,2}, {2,3}, {6,4}, {7,4}, {7,5}];
init {
// Place the food on the grid
ask world_cell grid_at {food_x, food_y} {
is_food <- true;
}
// Place the obstacles on the grid
loop pos over: obstacle_positions {
ask world_cell grid_at pos {
is_obstacle <- true;
}
}
write "Welcome to the Smart Forager!";
write "Grid size: " + grid_size + "x" + grid_size;
write "Food location: (" + food_x + ", " + food_y + ")";
write "Number of obstacles: " + length(obstacle_positions);
}
grid world_cell width: 10 height: 10 neighbors: 4 {
bool is_food <- false;
bool is_obstacle <- false;
rgb color <- #white update: is_obstacle ? rgb(60, 60, 60) : #white;
}
experiment smart_forager type: gui {
output {
display "Grid World" {
grid world_cell border: #lightgray;
graphics "food" {
ask world_cell where each.is_food {
draw circle(5) color: rgb(50, 180, 50);
}
}
}
}
}