example.py Overview

This module contains all default implementations of abstract classes. If this file is ran, it contains a main method which will run a default simulation with a DefaultSimulation, GameControlDriver and a Car.

DefaultSimulation

In the example.py module there exists a DefaultSimulation class which is able to run a simulation with minimal features/text/buttons. It is simply capable of adding a compatible vehicle with a driver to move around the track. Once the car crashes within the default simulation, the car will reset and the iteration count increased by 1.

Car

In the example.py module there exists a Car class which contains a barebones implementation of a functional car.

Car features:

• Acceleration
  • Acceleration multiplier can be set when initializing the car, this will change how fast the car speeds up
• Braking
  • The braking system slows the car down using the acceleration multiplier.
• Turning
  • The turning angular speed increases as the car velocity increases, as this is similar (kind of) to how cars would work if a car had perfect traction.

GameControlDriver

In the example.py module there exists a GameControlDriver class which allows the user to control the vehicle using the arrow keys on the keyboard.

genetic.py

The genetic module holds all simulators, cars and drivers related to running the genetic algorithm.

An example main() method is provided in the module and steps are outlined in comments.

    # the number of cars within a particular batch
    BATCH_SIZE = 100
    # the number of cars to render on the screen at once
    MINI_BATCH_SIZE = 20
    # step 1: generate cars, and create a new car set
    genetic_cars = GeneticCar.generate_cars(BATCH_SIZE, params=None)
    car_set = GeneticCarSet(genetic_cars, MINI_BATCH_SIZE)
    # step 2: generate genetic simulation
    simulation = GeneticAlgorithmSimulation(
        debug=True,
        fps=None,
        num_episodes=None,
        cars=car_set,
        track_offset=(0, 0),
        batch_size=BATCH_SIZE,
        mini_batch_size=3,
        caption="Genetic Algorithm Simulation"
    )
    # step 3: generate sensors and attach to the cars
    sb = SensorBuilder(
        depth=500,
        sim=simulation,  # ignore warning, simulations are compatible
        default_value=None,
        color="random",
        width=2,
        pointer=True
    )
    sensor_batch = sb.generate_sensors(sensor_range=(-90, 90, 10), num_sets=BATCH_SIZE)
    # Attach the sensors to the cars
    car_set.initialize_sensors(sensor_batch=sensor_batch)
    # calculate inputs and output size
    num_inputs = car_set.get_external_inputs() + sb.num_sensors
    num_outputs = Car.get_num_outputs()
    # step 4: initialize the drivers given the input/output numbers and put them in the cars
    initial_drivers = GeneticAlgorithmDriver.generate_drivers(BATCH_SIZE, num_inputs, num_outputs, epsilon=.50, load_latest=True)
    car_set.initialize_drivers(drivers=initial_drivers, simulation=simulation)
    # step 5: simulate!
    simulation.simulate()

Other features for the genetic algorithm, such as headless mode for processing is not supported at this time.

qlearn.py

The qlearn module holds all relevant classes for the Q-Learning Agent. This includes

• Experience - a data class responsible for encapsulating an event and its relevant properties
• ReplayMemory- responsible for storing/managing all experiencing and serving experience batches to the agent
• QLearningParams - class responsible for storing relevant reward functions/parameters QLearningAgent - Agent subclass responsible for driving the vehicle

A default implementation of a Q-Learning driver is implemented in example.py.

Future Releases

• Support for saving Q-Learning Models from json
• More tracks to test Genetic/QL models on
• Testing Suite to manage batches of simulations

More enhancements listed through GitHub issue tracking