Chahdnezhari/darija-ngram-model

# Darija N-gram Language Model ## Overview This project implements a probabilistic n-gram language model trained on a Darija corpus. ## Method - Data loaded from multiple text sources - Text cleaned and tokenized - Sentences extracted line-by-line - Train/validation split (90% / 10%) - Models trained: unigram, bigram, trigram - Laplace smoothing applied ## Results | Model | Perplexity | |--------|-----------| | 1-gram | 3558.91 | | 2-gram | 19638.79 | | 3-gram | 44591.97 | ## Conclusion The unigram model achieved the lowest perplexity and performed best on the validation set. Higher-order models suffered from data sparsity due to limited context coverage. ## How to Run ```bash python3 ngram_darija_project.py