MoloudAs/kinyarwanda-semantic-representations

# Kinyarwanda Semantic Representations This repository contains a reproducible Jupyter notebook pipeline for creating two semantic representations for a balanced Kinyarwanda noun–verb word list: - 300-dimensional FastText vectors; - 768-dimensional contextual KinyaBERT vectors. The target vocabulary comes from the companion repository: ```text kinyarwanda-kigali-wordlist ``` That repository produces a final dataset containing: - 10,000 unique words; - 3,333 nouns; - 6,667 verbs; - 9,926 words with exact FastText vocabulary entries; - 74 words without exact FastText entries. ## Repository structure ```text kinyarwanda_semantic_representations/ ├── data/ │ ├── raw/ │ ├── intermediate/ │ └── final/ ├── notebooks/ ├── reports/ ├── .gitignore ├── README.md └── requirements.txt ``` ## Required inputs The following files are required in `data/raw/`: ```text data/raw/ ├── kigali_final_10000_wordlist.csv ├── kigali_aligned.csv └── fasttext/ ├── kin.bin ├── kin.bin.vectors.npy ├── kin.bin.vectors_vocab.npy └── kin.bin.vectors_ngrams.npy ``` ### Final word list `kigali_final_10000_wordlist.csv` contains the final 10,000-word noun–verb dataset produced by the word-list repository. ### Aligned Kigali corpus `kigali_aligned.csv` contains the 30,890 aligned Kigali transcriptions used to find sentence occurrences for the target words. ### FastText model The complete four-file FastText bundle is required to load the saved Kinyarwanda `FastTextKeyedVectors` object and extract exact stored vectors. The raw input files and model files are not redistributed through this repository. ## Required packages Install the required Python packages with: ```bash python -m pip install -r requirements.txt ``` The main dependencies include: ```text pandas numpy scipy gensim matplotlib tqdm torch transformers jupyter ``` ## Notebook pipeline Run the notebooks in numerical order. ### 1. Extract FastText semantic vectors `01_extract_fasttext_semantic_vectors.ipynb` Thi …