GO-AI-CORPORATION/goai-bench

Open benchmarking toolkit for evaluating NLP models on low-resource Burkinabè languages. Open benchmarking toolkit for evaluating NLP models on low-resource Burkinabè languages, developed by **GO AI Corporation**. Currently focuses on **Mooré** and **Dioula** across three evaluation tasks: **Machine Translation (MT)**, **Automatic Speech Recognition (ASR)**, and **Text-to-Speech (TTS)**. **Public rankings and results** are on the GO AI Bench Leaderboard (Hugging Face Space). If your question is not covered here or in the docs below, open a GitHub issue or reach us via goaicorporation.org. **Documentation** (detailed guides): **English** — la même documentation est disponible **en français** sous `docs/fr/` (voir l’index docs/README.md). ## Table of Contents - Updates - Supported Languages - Supported Tasks - Architecture (overview) - Setup - Usage (quick examples) - Python API - Roadmap - Citation - Acknowledgments - License --- ## Updates - [04/2026] **Major update:** released **GO AI Bench v0.1.0** (this toolkit) and **GO AI Bench Leaderboard v0.1.0** — browse Mooré and Dioula results for ASR, TTS, and MT on the Space. --- ## Supported Languages | Language | ISO 639-3 | HF Code | Resource Level | MT | ASR | TTS | |----------|-----------|---------|----------------|----|-----|-----| | Moore | `mos` | `mos_Latn` | Low | Yes | Yes | Yes | | Dioula | `dyu` | `dyu_Latn` | Low | Yes | Yes | Yes | ## Supported Tasks | Task | Primary Metric | Baselines | Description | |------|---------------|-----------|-------------| | **MT** | chrF++ | NLLB-200 (3.3B, 1.3B, 600M) | Machine Translation (FR target) | | **ASR** | WER | Whisper, MMS | Automatic Speech Recognition | | **TTS** | UTMOS | MMS-TTS | Text-to-Speech (naturalness + intelligibility) | --- ## Architecture (overview) GO AI Bench wraps each model in a **provider** (`MTProvider`, `ASRProvider`, `TTSProvider`). YAML under `configs/datasets/` describes Hugging Face datasets and **benchmark groups**; `DataLoader` loads samples from the Hub; evaluators in `tasks/` compute metrics; `ResultWrite …