MoloudAs/kinyarwanda-kigali-wordlist

# Kinyarwanda Kigali Wordlist Pipeline This repository contains a reproducible Jupyter notebook pipeline for creating a balanced 10,000-word Kinyarwanda noun–verb dataset from the Kigali portion of a larger speech corpus. ## Data sources ### Speech corpus The source data come from the Kinyarwanda Automatic Speech Recognition — Track B dataset, released by **Digital Umuganda** with funding from the **Gates Foundation**. This project uses the `transcription` column from the training data. Each transcription provides the sentence context supplied to the Kinyarwanda morphological analyzer and POS tagger. The tagged words are then extracted, cleaned, and filtered to create the word list. The original Kaggle data are not redistributed in this repository. ### Kinyarwanda POS tagging The POS-tagged text was prepared using the Kinyarwanda morphological analyzer and POS tagger associated with KinyaBERT. Relevant references: - KinyaBERT implementation - DeepKIN toolkit - KinyaBERT paper ## Final dataset The main output is: ```text data/final/kigali_final_10000_wordlist.csv ``` It contains: - 10,000 words - 3,333 nouns - 6,667 verbs - 9,926 exact FastText matches - 74 words without an exact FastText match - 0 duplicate words The final CSV contains: ```text word original_forms pos_tags n_pos_tags total_frequency pos_tag_frequencies syllabification cv_pattern cv_syllables n_syllables fasttext_exact_match ``` ## Required packages ```text jupyter pandas gensim ipython ``` Install them with: ```bash python -m pip install -r requirements.txt ``` ## Notebook pipeline Run the notebooks in numerical order. ### 1. Extract the Kigali corpus `01_extract_kigali_corpus.ipynb` This notebook: - loads the cleaned training data; - checks the expected columns and dimensions; - normalizes the `location` column; - keeps only rows whose location is Kigali; - preserves the original columns and row order; - saves and validates the Kigali-only corpus. Expected result: - 180 …