myriamlmiii/darija-word2vec-nlp

Word2Vec Skip-gram embeddings for Moroccan Darija trained on 3M sentences - advancing NLP for low-resource Arabic dialects with neural language models # Darija Word Embeddings: Neural NLP for Low-Resource Arabic Dialects A neural word embedding system for Moroccan Darija using Word2Vec Skip-gram with negative sampling, trained on 3 million sentences from a curated 8.7M sentence corpus to advance natural language processing capabilities for under-resourced North African languages. ## 🎯 Research Motivation Moroccan Darija, spoken by over 30 million people, remains critically under-resourced in NLP research despite being the primary language of Morocco. This project addresses the fundamental challenge of creating distributional semantic representations for dialectal Arabic, enabling computational linguistics research and practical applications for Moroccan digital infrastructure. **Research Gap:** While Modern Standard Arabic has substantial NLP resources, Maghrebi dialects like Darija lack basic tools including word embeddings, trained language models, and annotated corpora. **Contribution:** Large-scale Word2Vec embeddings for Darija trained on web-curated corpus, establishing baseline semantic representations for downstream NLP tasks in Moroccan Arabic dialect processing. ## 🛠️ Technical Architecture ### Neural Language Model **Word2Vec Skip-gram with Negative Sampling:** - **Architecture:** Predicts context words given target word - **Optimization:** Negative sampling (k=10) for computational efficiency - **Complexity Reduction:** O(V) → O(k log V) per training example - **Training Objective:** Maximize log probability of context words **Mathematical Foundation:** ``` Objective: maximize Σ log σ(v'_c · v_w) + Σ_i log σ(-v'_ni · v_w) where σ = sigmoid, v_w = target word vector, v'_c = context vector ``` **Why Skip-gram Over CBOW:** - Superior performance on rare words and morphological variants - Better semantic relationship capture in low-resource settings - Handles morphologically rich languages more effectively - Proven effectiveness for dialectal and under-resourced languages ### Computational Infra …