Development Environment - wbap/WM_Hackathon GitHub Wiki

The WM_Hackathon project contains a basic agent that can solve the match the sample task. It is provided as a basis for improved agents. This page explains how you can easily set up a development environment for this project. There are two approaches, you can use a Docker container, or set up a Conda environment. If you are using a Mac, the Docker container does not work fully i.e. it is not possible to view the graphical user interface. Therefore, if you want to play the challenge manually and you are using a Mac, you should choose the Conda option.

Table of Contents

  1. Using Docker for your Development Environment
  2. Set up your own Development Environment
  3. GPU Usage

Using Docker for your Development Environment

Requirements

  • Docker

    • Follow instructions on website to set it up locally (depends on your environment)
    • Windows
      • Docker Desktop for Windows
        • WSL2 “Windows Subsystem for Linux” (Follow instructions: Manual Installation Step 1 - 5)

  • X11 if you want to display a GUI

  • NVIDIA Docker Package (if you have an NVIDIA GPU)

    • NOTE: This will not apply if you are using a Mac / Windows.
    • Follow instructions on website

  • Source code

Background

This deployment solution allows you to run the software on any machine running docker. The environment is set up for you already. You will have a local copy of the code for development with your preferred tools. It will be mounted inside the container for running.

There is a Docker Image called wb_wm_hackathon available on Docker Hub. It contains the environment setup to run the software. When you run a command using the scripts in this folder, the Image will be automatically downloaded, cached on your local machine, and used to create a container. Your commands will then be run inside the container.

The container allows you to utilise the GPU if your machine has one (Linux Only). It will also by default, utilise X11 on your machine to display the GUI, which is relevant when you want to play the game as a human and see the screen (as opposed to letting the agent play the game without displaying anything to the user). Note that this is not supported for Macs. If you are using a Mac, you can easily create a conda environment to run the software (more below)

Getting Started

  1. Clone the code

    • WM_Hackathon repository
    • cerenaut-pt-core repository

  2. Running the code

    • Navigate to the repository root folder
    • All you need to do is pass a given command to the script deployment/run-docker.sh to run it in the container.
    • The full usage instructions are: run-docker.sh [absolute path to wm code] [absolute path to cerenaut-pt-core code] [boolean for GPU or not] [command and params to run in container]
    • Windows
      • Use the script deployment/run-docker.ps1 on Windows PowerShell.

    Examples:

    deployment/run-docker.sh ~/Dev/WM_Hackathon ~/Dev/cerenaut-pt-core false python keyboard_agent.py m2s-v0 configs/m2s_env.json

    See Getting Started for examples of other commands that can be run e.g. to pretrain the visual cortex or to train the agent to complete the M2S task.

    NOTE: If you are on Ubuntu, you'll need to use sudo

    NOTE: With deployment/run-docker.ps1 on Windows PowerShell, ~ cannot be used. Please use the full path.

  3. Development

    • While running the code, you can monitor and debug using Tensorboard.
    • The host port localhost:6006 is mapped to 6006 in the container, which is the default port for Tensorboard.
    • You will need to run Tensorboard first, which can be accomplished by getting a shell in the container.
    • See Docker documentation. In brief:
      • see which containers are running with docker ps
      • copy the container ID, then
      • docker exec -it [container ID] bash

Custom Container

You can modify and build your own dev environment. First modify the Dockerfile to your spec. Then use the script build-docker.sh [image name] to build a local copy of the image.

You can then modify the run-docker.sh script to refer to your custom image name, and run as above.

Using Tensorboard

The suggested way to use Tensorboard, is to run it locally on the host, not in the Docker container. That way, you have more control over how it is run, and it won't be shut down when you stop the container.

The local source code folder is mounted into the Docker container and the outputs will be in the /run subfolder. Pass this in as the location of summary files for Tensorboard.

i.e. tensorboard --logdir=WM_Hackathon/run

Note that it is also possible to run Tensorboard in the Docker container. The run-in-docker.sh script already maps the default Tensorboard port to the localhost. To run it, you will need to manually get a shell into your Docker container or modify the run-in-docker.sh scripts.

Set up your own Development Environment

This is useful if you use a Mac and you want to see the PyGame GUI (which is a special case that does not work). The instructions show how you can create your own Conda environment for development and running.

Requirements

Getting Started

  1. Clone the code

    • WM_Hackathon repository
    • cerenaut-pt-core repository

  2. The file WM_Hackathon/deployment/environment.yaml defines all of the dependencies that are required. You can use it to create a conda environment with the command below, see instructions here.:

conda env create -f environment.yml
  1. Then, when you want to dev and run, change into the environment. This is a platform specific command. It is usually:
conda activate wm_env

On same systems you may need to use:

source activate wm_env
  1. Install custom Gym environment: You need to pre-install the custom Gym environments contained in this repository.
cd WM_Hackathon
pip install --ignore-installed --no-deps -e .
  1. Finally, you need to setup a dependency, the project cerenaut-pt-core by following the README.md. In brief, navigate to the folder and then use the following command:
python setup.py develop

GPU Usage

Pre-training with PyTorch

For any non-agent model pre-training (e.g. vision components), the framework will automatically detect and use the GPU when available. You can verify that PyTorch can detect and use the GPU in your environment (Docker or otherwise), by running python -c 'import torch; print(torch.cuda.is_available())' which should return True if PyTorch can utilise the GPU.

If this returns False and you have a functioning GPU, this could mean that the CUDA versions are not aligned with the relevant PyTorch version or your system's NVIDIA driver. Please report any issues you encounter on the Slack.

Training Agents with Ray

Ray does not automatically detect and utilise the GPU, and it has to be explicitly specified in your agent configuration file. To modify these settings, you can update the agent_av.json file (or equivalent) and modify the agent section in JSON file.

Set num_gpus to the number of GPUs available in your environment, and set num_gpus_per_worker to number of workers you would like to deploy per GPU. The setting of these parameters will entirely depend on your environment and system. Typically, algorithms such as A3C, will be more limited by the CPU rather than the GPU.

During evaluation, the competition will be using CPU only for inference, so please ensure that the agent_av_pretrained.json file in your final submission to the competition is only configured for the CPU.