03. Supervised Learning - AlphaGoECM/AlphaGo GitHub Wiki

Two neural networks are part of AlphaGo : the policy and the value network. Only the policy network is train under supervised learning. This first training aims at reaching a certain point of "expertise" from high-level human players.

Neural Network

According to DeepMind's paper in Nature, their architecture was the following : "The input to the policy network is a 19x19x48 image stack consisting of 48 feature planes. The first hidden layer zero pads the input into a 23x23 image, then convolves k filters of kernel size 5x5 with stride 1 with the input image and applies a rectifier nonlinearity. Each of the subsequent hidden layers 2 to 12 zero pads the respective previous hidden layer into a 21x21 image, then convolves k filters of kernel size 3x3 with stride 1, again followed by a rectifier nonlinearity. The final layer convolves 1 filter of kernel size 1x1 with stride 1, with a different bias for each position, and applies a softmax function. The match version of AlphaGo used k = 192, 128, 256 and 384 filters."

For facilities, because we don't have such a subtil hardware like Google, we used only 32 filters and 4 hidden layers. Here are our result after training : %A COMPLETER%

Training

From the shell, here are all the commands you can run in order to train our neural network over a hdf5 dataset :

    parser.add_argument("--train_data","-t", help="A hdf5 file of training data")
    parser.add_argument("--proportion", "-p", help=" %\of data used in the data file. Default: 100%", type=int, default=100)
    parser.add_argument("--out_directory","-o", help="directory where metadata and weights will be saved")
    parser.add_argument("--model", "-mo", help="load a hdf5 model file", default=None)
    parser.add_argument("--layers_nb", "-L", help="Total number of intern Conv2D layers, Default: 11 (+2 others)", type=int, default=11)
    parser.add_argument("--board_size", "-s", help="Size of the go board", type=int, default=19)
    parser.add_argument("--batch", "-B", help="Size of training data batches. Default: 16", type=int, default=16)
    parser.add_argument("--epochs", "-E", help="Total number of iterations on the data. Default: 4", type=int, default=4)
    parser.add_argument("--epoch-length", "-l", help="Number of training examples considered 'one epoch'. Default: # training data", type=int, default=None)
    parser.add_argument("--learning-rate", "-r", help="Learning rate - how quickly the model learns at first. Default: .03", type=float, default=.03)
    parser.add_argument("--decay", "-d", help="The rate at which learning decreases. Default: .0001", type=float, default=.0001)
    parser.add_argument("--verbose", "-v", help="Turn on verbose mode", default=False, action="store_true")
    parser.add_argument("--generator", "-gen", help="Turn on generator data mode", default=False, action="store_true")
    parser.add_argument("--symmetries", help="Comma-separated list of transforms, subset of noop,rot90,rot180,rot270,fliplr,flipud,diag1,diag2", default='noop,rot90,rot180,rot270,fliplr,flipud,diag1,diag2')  # noqa: E501

Here is an example of command for training : python -m SL_training -t /donnees/allgames.hdf5 -p 10 -o ./output/ -L 4 -E 20 -v