HanneWhitt/mancalazero

An implementation of AlphaZero for the ancient African game, Mancala! # mancalazero An implementation of AlphaZero for the ancient African game, Mancala! Implemented using the methods described in: "Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm", Silver et al., DeepMind, 2018 "Mastering the game of Go without human knowledge", Silver et al., DeepMind, 2017 The implementation is from the ground up, with fully original code and the only non-standard Python packages used being numpy, torch, and networkx/graphviz for MCTS tree visualisation. ### Scripts: * ```gamestate.py``` contains an abstract base class (ABC) ```GameState``` intended as a framework for the implementation of a wide range of classical games; I hope that this strategy will make it possible to extend this project quite simply to other games once it works for Mancala. A simple test game, involving flipping a coin, can be seen in ```tests/gamestate_test.py```; ```mancala.py``` implements the rules of Mancala in several variations, and allows access to the state of a game as a vector of length 16, with features for the number of stones in each of the 14 spaces on the board, the current player, and the current turn number. * ```mancalanet.py``` uses PyTorch to build a neural network suitable for use in AlphaZero, taking as input the state of a game, and outputing a policy vector and value estimate. The policy vector is a probability distribution describing the networks' view of which moves are more or less promising, and the value provides the networks' estimate of win probability from the current position. The AlphaZero loss function is implemented in ```loss.py```, and uses the outcomes of real games to train the value component, the MCTS search probabilities to train the policy component, and a customised L2 weight regularisation as a measure against overfitting. * ```MCTS.py``` implements Monte Carlo Tree Search, an algorithm which extends the neural network's strength by using it to look ahead and intelligently sample possi …