Eyaa-Tom 26, Yodi-Mantissa and Lom Bench: A Community Benchmark for TTS in Local Languages

Most of the more than 40 languages spoken in Togo lack the data and tools necessary for modern natural language processing (NLP) applications. We present an extension of previous work by introducing new datasets, improved models, and a community-driven evaluation benchmark for text-to-speech (TTS). We expanded the Eyaa-Tom multilingual corpus with additional speech data (e.g. 26.9k recordings, 30.9 hours) across 10 local languages and incorporated Mozilla Common Voice contributions (64.6k clips, 46.6 hours) for Adja, Nawdm, Mina, Tem to strengthen automatic speech recognition (ASR) and speech synthesis. We detail how community contributors (including collaboration with a national TV journalist) helped collect and validate the Kabiyɛ and French text, with an ethical compensation model in place. We also try to compare the performance of a few models in these datasets, we fine-tuned state-of-the-art models in these data for ASR, OpenAI Whisper and faster-whisper were benchmarked achieving improved word error rates after fine-tuning; for machine translation, we fine-tuned Meta's NLLB-200 model in 11 local languages, which produced significant BLEU/METEOR gains especially in Ewɛ, and Kabɩyɛ. To evaluate TTS, we introduce Lom Bench, a new community-based benchmark where native speakers rate synthetic speech. The preliminary results from Lom Bench indicate promising naturalness in Ewɛ and Kabɩyɛ TTS, although further data is needed.