k1ndofthought/Amharic-word2vec-vs-afroxlmr

Undergraduate thesis project comparing Word2Vec and Afro-XLM-R for Amharic semantic similarity, emphasizing efficiency and accuracy trade-offs in low-resource NLP. # Word2Vec vs. Afro-XLM-R for Amharic Semantic Similarity Undergraduate thesis project (Sivas Cumhuriyet University, Dept. of Statistics & Computer Science, 2026) comparing a static word embedding model (**Word2Vec**) against a contextual multilingual transformer (**Afro-XLM-R**) on Amharic semantic textual similarity — evaluated both for **accuracy** and for **computational efficiency**, framed through a **Green AI** lens. > Amharic is a morphologically rich, low-resource language with very few ready-made NLP tools, labeled datasets, or pretrained models. This project asks: for a low-resource language *and* low-resource hardware, is a lightweight 2013-era embedding model still a reasonable choice against a modern transformer? ## Results A hand-built evaluation set of 50 Amharic sentence pairs (human-rated 0–5 for semantic similarity) was scored by both models using cosine similarity over mean-pooled sentence embeddings, then correlated against human judgments. | Model | Spearman ρ | p-value | Pearson r | Inference time (50 pairs) | Model size | |---|---|---|---|---|---| | Word2Vec (Skip-gram, 200d) | 0.360 | 0.131 | 0.368 | 0.002 s | 2.4 MB | | Afro-XLM-R (feature extraction) | 0.410 | 0.081 | 0.368 | 3.91 s | 1,060.7 MB | **Word2Vec is ~1,600x faster and ~440x smaller on disk**, at the cost of a modestly lower rank correlation with human judgment. Afro-XLM-R's contextual representations track human similarity judgments slightly better, but the accuracy gain is small relative to the jump in compute and memory footprint — illustrating the accuracy/efficiency trade-off that Green AI research is concerned with. ## Method - **Word2Vec**: Skip-gram architecture (`vector_size=200, window=5, min_count=5, epochs=30`), trained from scratch on a corpus of Amharic news sentences using Gensim. Sentence vectors are the mean of in-vocabulary word vectors. - **Afro-XLM-R** (`Davlan/afro-xlmr-base`): used in feature-extraction mode (no fine-tuning, due to lack of label …