KamelTouati/algerian-dialect-review-sentiment-analysis-Djezzy

# Algerian Dialect Review Sentiment Analysis (Djezzy) Excited to share my journey and results on Algerian Dialect Sentiment Analysis for Djezzy! Understanding customer feedback is crucial, especially in nuanced local dialects. This project focused on fine-tuning DziriBERT, the first Transformer model pre-trained specifically for the Algerian Dialect, to classify customer reviews for Djezzy. ## Project Evolution 1️⃣ Phase 1: Initial Model & Baseline - Utilized the open "Algerian Dialect Review for Sentiment Analysis" dataset from Kaggle (~5.3k samples). - Fine-tuned DziriBERT by Alger-IA. - Result: Achieved a respectable initial test accuracy of ~82%. - Observation: Good start, but class imbalance (especially for positive sentiment) was noticeable. 2️⃣ Phase 2: Data Enhancement & Semi-Supervised Labeling - To improve robustness, I augmented the dataset by scraping ~4,000 additional Djezzy-related comments from public Facebook posts. - Leveraged the V1 model for initial annotation, followed by careful manual verification to ensure data quality. - Dataset size grew to ~9,400 samples. - Result: Test accuracy improved to ~84%, with validation accuracy reaching ~85.5%. - Observation: More data helped, but the positive class was still significantly smaller, impacting its recall and F1-score. 3️⃣ Phase 3: Tackling Class Imbalance with Augmentation & Weighted Loss - Focused on balancing the dataset by augmenting the underrepresented positive class. - Employed techniques like simulated back-translation, random insertion/deletion/swaps. - The dataset expanded to ~12,300 effective samples. - Implemented class weights during training to give more importance to minority classes. ### Results - 🚀 Test Accuracy boosted to ~87%! - 📈 Macro F1-score on the test set improved to ~0.88. - Most significantly, the F1-score for the Positive class surged from ~0.80 (in V2) to an excellent ~0.95! This means the model is now much better at identifying positive feedback. This iterative p …