ianktoo/tafsiri

Generate & evaluate fine-tuning-ready African-language translation datasets, powered by Daraja AI's Babel models. Multi-signal eval (confidence, back-translation, LLM-as-judge) + SQLite persistence. # tafsiri Turn raw text into **fine-tuning-ready translation datasets** for African languages, with quality scores you can trust. `tafsiri` ("translation" in Swahili) runs a simple pipeline: ``` source text → translate (Babel / LLM engine) → evaluate → score → structured training data + an eval report ``` It translates your text into one or more African languages, evaluates each translation several independent ways, scores it, and writes out fine-tuning data plus a report telling you whether the quality is good enough to use. Everything is persisted to SQLite so nothing is lost between sessions. > **Name note:** "Daraja" is also Safaricom's M-PESA API. This project targets > the Daraja AI **translation** API (`api.daraja.ai`), not M-PESA. ## Background African languages are spoken by well over a billion people, yet most remain **low-resource** for machine translation: training corpora are scarce, and general-purpose models - built largely on English and other high-resource languages - tend to translate them inconsistently. Purpose-built efforts like Daraja AI's *Babel* models aim to close that gap. For many real applications (emergency response, healthcare, finance) a translation isn't useful unless you can tell whether it's **trustworthy** - a fluent-sounding but subtly wrong translation can be worse than none. Yet quality measurement for low-resource African languages is itself underdeveloped. `tafsiri` is a small **research harness** for exactly this. It holds a reproducible evaluation pipeline fixed and lets you vary the inputs: - **Swap the translation engine** (a purpose-built API like Babel, or a general LLM) and compare quality on identical inputs. - **Combine evaluation signals** - the engine's own confidence, back-translation round-trips, and an LLM-as-judge - into a single score and rating. - **Run across domains and languages** using the bundled `samples/` datasets, and get per-language / per-speaker score breakdowns. - **Produce …