ForagerRL_step1 - gama-platform/gama GitHub Wiki

The Smart Forager — Step 1: The Grid World

By Killian Trouillet

This is Part 1 of the internal Reinforcement Learning tutorial in GAMA.

Tutorial Overview

Step Title Concepts
1 The Grid World grid, global, experiment, display
2 The Forager Agent species, reflex, aspect, switch/match
3 Rewards and Episodes Reward function, episode management, monitor
4 The Q-Table map<string, float>, state representation
5 Q-Learning Algorithm Epsilon-greedy policy, Bellman equation
6 Charts, Test Mode and Visualization chart, heatmap, training stop, automatic test

Step 1: The Grid World

Content

This first step illustrates how to create a grid environment in GAMA. We define a 10×10 grid with round food and rectangular obstacles.

Formulation

  • Definition of global parameters for grid size, food position, and obstacle positions
  • Definition of a world_cell grid with is_food and is_obstacle attributes
  • Display of the grid with colored cells: dark gray for obstacles, white for empty, and a green circle for the food

Model Definition

Global section

The global section defines the world parameters. We store the grid size, the food coordinates, and a list of obstacle positions as point values.

global {
    int grid_size <- 10;
    int food_x <- 9;
    int food_y <- 9;
    list<point> obstacle_positions <- [{2,2}, {3,2}, {2,3}, {6,4}, {7,4}, {7,5}];
  • int grid_size <- 10: Defines the size of our grid (10 columns × 10 rows).
  • list<point>: A list of coordinate pairs. Each {x, y} represents the position of an obstacle.

Initialization

In the init block, we configure the grid cells by setting their is_food and is_obstacle attributes. The operator grid_at allows us to access a specific cell by its coordinates.

    init {
        ask world_cell grid_at {food_x, food_y} {
            is_food <- true;
        }
        loop pos over: obstacle_positions {
            ask world_cell grid_at pos {
                is_obstacle <- true;
            }
        }
    }
}
  • ask world_cell grid_at {x, y} { ... }: Selects the grid cell at coordinates (x,y) and executes the block on it.
  • loop pos over: list { ... }: Iterates over each element of the list.

Grid definition

A grid in GAMA is a special type of species where each cell is an agent. We define two boolean attributes (is_food, is_obstacle) and a dynamic color attribute that uses the update facet to reflect the cell state at every simulation step.

grid world_cell width: 10 height: 10 neighbors: 4 {
    bool is_food <- false;
    bool is_obstacle <- false;
    rgb color <- #white update: is_obstacle ? rgb(60,60,60) : #white;
}
  • width: 10 height: 10: Creates a 10×10 grid of cells.
  • neighbors: 4: Uses Von Neumann neighborhood (up, down, left, right only — no diagonals).
  • update: facet: Used here to color obstacles only. The food will be drawn separately.

Experiment

The experiment block defines how to run and visualize the simulation.

experiment smart_forager type: gui {
    output {
        display "Grid World" {
            grid world_cell border: #lightgray;
            graphics "food" {
                ask world_cell where each.is_food {
                    draw circle(5) color: rgb(50, 180, 50);
                }
            }
        }
    }
}
  • type: gui: A graphical experiment with a user interface.
  • grid ... border: #lightgray: Displays the grid cells.
  • graphics "food" { ... } : A graphics layer to draw objects manually. We ask every food cell to draw a circle at its location.

Complete Model

/**
* Name: SmartForager - Step 1: The Grid World
* Author: Killian Trouillet
* Description: This first step creates a simple 10x10 grid environment 
*              with food (green cell) and obstacles (dark gray cells).
* Tags: reinforcement-learning, grid, tutorial
*/

model SmartForager

global {
	int grid_size <- 10;
	int food_x <- 9;
	int food_y <- 9;
	list<point> obstacle_positions <- [{2,2}, {3,2}, {2,3}, {6,4}, {7,4}, {7,5}];
	
	init {
		// Place the food on the grid
		ask world_cell grid_at {food_x, food_y} {
			is_food <- true;
		}
		// Place the obstacles on the grid
		loop pos over: obstacle_positions {
			ask world_cell grid_at pos {
				is_obstacle <- true;
			}
		}
		write "Welcome to the Smart Forager!";
		write "Grid size: " + grid_size + "x" + grid_size;
		write "Food location: (" + food_x + ", " + food_y + ")";
		write "Number of obstacles: " + length(obstacle_positions);
	}
grid world_cell width: 10 height: 10 neighbors: 4 {
	bool is_food <- false;
	bool is_obstacle <- false;
	rgb color <- #white update: is_obstacle ? rgb(60, 60, 60) : #white;
}

experiment smart_forager type: gui {
	output {
		display "Grid World" {
			grid world_cell border: #lightgray;
			graphics "food" {
				ask world_cell where each.is_food {
					draw circle(5) color: rgb(50, 180, 50);
				}
			}
		}
	}
}

⚠️ **GitHub.com Fallback** ⚠️