solalekan/tesseract-yoruba-traineddata

yor.traineddata - Improved Yoruba traineddata for Tesseract OCR # yor.traineddata - Improved Yoruba Traineddata for Tesseract OCR ### 📝 Purpose For decades, progress in Yoruba digital text processing and preservation has been significantly slowed by the lack of accurate Optical Character Recognition (OCR) resources. Researchers, linguists, and historians have faced substantial barriers when scanning Yoruba documents — particularly due to poor OCR support for complex characters and diacritics. Many are forced to rely on expensive commercial services just to digitize Yoruba texts with acceptable accuracy. This repository offers a powerful, open-source alternative: an **improved `yor.traineddata`** for Tesseract OCR, designed specifically to enhance the accuracy of Yoruba text recognition and support long-term development of Yoruba in the digital world. --- ### ✅ Improvements Over the Official Model The official Yoruba OCR model included in Tesseract’s `tessdata_best` suffers from **critical limitations in recognizing combined diacritic characters**, especially when both an underdot and a tonal accent are present. #### Observed problems in the official model: - ✅ Scans basic tonal characters (e.g., `È`, `Ó`) - ✅ Scans underdot characters (e.g., `Ṣ`, `Ẹ`) - ❌ Fails or produces poor accuracy for combined characters with both underdot and tone (e.g., `Ẹ́`, `Ọ̀`) #### Root cause: Our research suggests that the original model was likely trained with **unnormalized (non-NFC)** text. This causes issues during `.box` file generation, where decomposed character sequences are not visually or structurally aligned in OCR training. As a result, the model fails to recognize composite forms reliably. #### Our solution: We designed a **custom normalization and character composition system**: - All training texts were normalized to **NFC (Normalization Form C)**. - A script was written to **automatically reconstruct complex character compositions** in `.box` files. - Each base character was merged with its respective diacritic and/or under …