MALIBA-AI/bambara-text-normalization

Bambara Text Normalizer Text Normalization & ASR Evaluation Framework for Bambara (Bamanankan) Installation • Normalization • ASR Evaluation • Modes • CLI • Linguistics • References --- ## Purpose This tool serves **two complementary purposes** for Bambara language processing: | Purpose | Description | |---------|-------------| | **Text Normalization** | Standardize Bambara text for any downstream NLP task (TTS, MT, NER, etc.) | | **ASR Evaluation** | Fair WER/CER computation that accounts for valid orthographic variation | > [!NOTE] > Bambara orthography allows variation: the same utterance can be written as `k'a ta` or `ka a ta` both are correct. Without normalization, evaluation metrics unfairly penalize models for human writing inconsistencies rather than actual recognition errors. --- ## Installation ```bash # Core library only pip install bambara-text-normalizer # Install with CLI support pip install "bambara-text-normalizer[cli]" # Install everything pip install "bambara-text-normalizer[all]" ``` ```bash pip install git+github.com ``` --- ## Text Normalization ```python from bambara_normalizer import normalize # Default: expand contractions normalize("⁠Ne k’a ma ko ayi") normalize("⁠K’ale t’a fɛ k’a kɛ") normalize("⁠K’i k’i janto i yɛrɛ la") # Contract mode: collapse expanded forms normalize("Ne ko a ma ko ayi", mode="contract") normalize("Ko ale tɛ a fɛ ka o kɛ", mode="contract") normalize("Ko i ka i janto i yɛrɛ la", mode="contract") # Preserve mode: don't touch contractions normalize("K’i k’i janto i yɛrɛ la", mode="preserve") ``` ### Custom Settings ```python from bambara_normalizer import normalize # Full control over normalization text = normalize( "Ka na son k’o k’a la", mode="expand", # "expand" | "contract" | "preserve" preserve_tones=False, normalize_legacy_orthography=True, lowercase=True, remove_punctuation=False, normalize_whitespace=True, …