GhanaNLP/ghana-corpus-builder

Streamlined access to high-quality corpora for 38 Ghanaian languages # Ghana Corpus Builder A toolkit and small Python library for **retrieving monolingual and parallel text corpora for Ghanaian languages**. Pull a monolingual corpus for any single Ghanaian language, or pair one with English, with another Ghanaian language, or with one of several other world languages. Output is clean, sentence-aligned CSV, ready for machine-translation training and NLP research. The corpora are hosted on HuggingFace at `ghananlpcommunity/ghana-corpus`. The library downloads only the files you actually use and caches them locally, so the repository itself stays lightweight. New languages pushed to the dataset are picked up automatically — no code change or update needed. --- ## What you can build | Corpus type | Example | Notes | |---|---|---| | Ghanaian ↔ English | Twi ↔ English | English ships cached; the default target | | Ghanaian ↔ Ghanaian | Twi ↔ Ewe, Ga ↔ Dagbani | align two local languages directly | | Ghanaian ↔ other language | Twi ↔ French, Ewe ↔ Arabic | French, Spanish, Portuguese, German, Italian, Arabic, Chinese, Swahili are cached | | Monolingual | all Twi sentences | any single language | Every language's text is aligned on a shared verse key, so **any** two languages can be turned into a parallel corpus by a simple join. That is what makes Ghanaian ↔ Ghanaian and Ghanaian ↔ other-language pairs possible. ## Dataset statistics **47 Ghanaian languages** with data, **1,289,419 parallel sentence pairs** in total (English-aligned). Sentence pairs = aligned verses available before de-duplication. | Language | Code | Sentence pairs | |---|---|--:| | Akuapem Twi / Asante Twi | twi | 185,554 | | Ewe | ewe | 92,544 | | Ga | gaa | 62,004 | | Fante | fat | 61,606 | | Dagbani | dag | 61,558 | | Hausa | hau | 61,548 | | Sehwi | sfw | 39,005 | | Dagaare | dga | 38,948 | | Kasem | xsm | 38,631 | | Bimoba | bim | 38,180 | | Dangme | ada | 38,142 | | Konkomba | xon | 38,062 | | Nzema | nzi | 36,195 | | Fulfulde; Maasina | ffm | 36,012 | | V …