v0.7.0 - QueensGambit/CrazyAra GitHub Wiki

Strength Evaluation v0.7.0

In the following, information is given about the strength evaluation after 1 million generated self-play games. The model (RISEv2) was initialized via supervised learning on human lichess crazyhouse games and it was evaluated after 50th model updates during reinforcement learning.

CrazyAra 0.7.0 (Model-OS-45) played Multi-Variant-Stockfish-x86_64-modern 2019-12-03 in 100 matches (TC 15min+10s).

Eight examples of these games have been included in the master thesis

  • Deep Reinforcement Learning for Crazyhouse by Johannes Czech.

50 random opening starting positions have been used out of a list of 81 openings. The positions have been provided by FM Optilink:

Hardware Setup

  • Memory (RAM): 31,4 GiB
  • Processor - AMD® Ryzen 7 1700 eight-core processor × 16
  • Graphics - GeForce GTX 1080 Ti/PCIe/SSE2
  • OS type - Ubuntu 18.04.3 LTS, 64-bit

UCI-Options

Multi-Variant-Stockfish (2019-12-03)

All default except:

  • Threads: 8
  • Hash: 4096

CrazyAra 0.7.0

  • Search_Type: MCTS
  • Context: gpu
  • Device_ID: 0
  • Batch_Size: 16
  • Threads: 2
  • Centi_CPuct_Init: 250
  • CPuct_Base: 19652
  • Centi_Dirichlet_Epsilon: 0
  • Centi_Dirichlet_Alpha: 20
  • Centi_U_Init: 100
  • Centi_U_Min: 100
  • U_Base: 1965
  • Centi_U_Init_Divisor: 100
  • Centi_Q_Value_Weight: 0
  • Centi_Q_Thresh_Init: 50
  • Centi_Q_Thresh_Max: 90
  • Q_Thresh_Base: 1965
  • Max_Search_Depth: 99
  • Centi_Temperature: 80
  • Temperature_Moves: 0
  • Centi_Temperature_Decay: 92
  • Centi_Node_Temperature: 200
  • Virtual_Loss: 3
  • Nodes: 15000000
  • Allow_Early_Stopping: True
  • Use_Raw_Network: False
  • Enhance_Checks: False
  • Enhance_Captures: False
  • Use_Transposition_Table: True
  • Use_TensorRT: True
  • Model_Directory: model_os_45/
  • Move_Overhead: 50
  • Centi_Random_Move_Factor: 0

Nodes per Second (NPS)

  • Multi-Variant-Stockfish: 6.7 Million NPS
  • CrazyAra 0.7.0: 7-10 K NPS

Result

Score of CrazyAra-0.7.0-Model-OS-45-batch-16-temp-200 vs stockfish-x86_64-modern  2019-12-03: 61 - 36 - 3 [0.625]
Elo difference: 88.7 +/- 70.2, LOS: 99.4 %, DrawRatio: 3.0 %

100 of 100 games finished.

All games can be downloaded here: