AustinGTI/voteTallyKE

An attempt to achieve more than 90% accuracy in tallying the vote forms for Kenya's 2022 presidential elections through Machine Learning # VoteTallyKE This repo provides a complete system that is capable of extracting handwritten vote tallies from images of the polling station electoral forms with up to 95% accuracy A custom neural network based on the YOLOv5 object detection architecture is used to read the handwritten digits and a separate neural network also based on YOLOv5 is used to locate the position of the QR code from which the location of the tallies can be calculated. As a backup, the paddleOCR character recognition engine is used to locate keywords on all forms that can be used to calculate the location of the tallies (this is only a backup as it is slower though more accurate) --- ## Training the neural networks ### Handwritten digits The first neural network was trained to detect handwritten digits accurately using custom training data that is generated using the popular MNIST digit dataset. The digits are placed in a 1000 by 1000 px image that replicates the appearance of that area of the electoral form as closely as possible. This is done by : * replicating the background pattern of the form * drawing horizontal gridlines around the digits * placing the digits in 4 lines of 3 digits each as is consistent with the 4 electoral candidates * Adding image artifacts common in the electoral form eg. stamp marks, * Adding random perlin noise and blur to the image 1000 such images are generated with the bounding locations of each digit stored in corresponding text files. The images are split into training, testing and validation datasets and a model is trained on them using the YOLOv5 nano architecture to recognize handwritten digits as they would appear in the forms. ## QR Code The entire form is too large and unpredictable to reliably read the correct digits on a regular basis. For this reason, the rotation,scale and position of the form in the image need to be calculated and accounted for in order to crop out the required area. This is done by training a neural network to …