Hodan-dev/somali-text-classification

Somali language text classification project # Somali Language Text Classification This project classifies Somali text into 10 categories: - Siyaasad - Ciyaaraha - Waxbarasho - Caafimaad - Ganacsi - Tiknoolaji - Diin - Amni - Madadaalo - Bulsho ## Dataset The cleaned dataset is stored in `cleaned_data.csv` and contains: - `processed_text` - `Title` The project uses around 5,000 rows and an 80/20 train-test split. Each category already has about 500 samples, so it is well above a 100-sample minimum per class. ## Project Structure ```text text clasification codex/ |-- cleaned_data.csv |-- backend/ | |-- server.js | |-- package.json |-- frontend/ | |-- src/ | |-- package.json |-- ml/ | |-- preprocess.py | |-- train_model.py | |-- app.py | |-- model.pkl | |-- model_lr.pkl | |-- model_nb.pkl | |-- vectorizer.pkl | |-- confusion_matrices.png ``` ## Model Training The training pipeline now compares two models using the same TF-IDF feature space: - Logistic Regression - Multinomial Naive Bayes The TF-IDF representation uses: - word n-grams `(1, 2)` - character n-grams `(3, 5)` Both models are trained on the same 80% training split and evaluated on the same 20% test split for a fair comparison. ## Evaluation Metrics Each model is evaluated with: - Accuracy - Precision - Recall - F1-score The script prints a readable comparison table and also generates a confusion matrix figure for both models. ## Baseline vs Final Model Multinomial Naive Bayes is included as a baseline model. It is fast and simple, which makes it a good reference point. Logistic Regression is kept as the final model when it performs best on the held-out test set, or when it matches the baseline and is preferred as the stronger default for sparse TF-IDF features because it can learn more flexible decision boundaries. To reduce misleading predictions on mixed-topic or ambiguous text, the API can return `Uncertain` when the model confidence is low or when the top two classes are too close. ## Saved Artifacts …