anrilombard/mzansi-text

MzansiText is a curated multilingual pretraining corpus for all eleven official South African languages. Languages: af, en, nso, sot, ssw, tsn, tso, ven, xho, zul, nbl Schema: { "text": "string", "lang": "string" } This repository contains the raw train, validation, and test text splits used for the MzansiLM pretraining release. The token distribution table below matches the paper-reported corpus statistics.