Souadzriouil/darijabert-lora-finetuning

This project focuses on fine-tuning DarijaBERT for Moroccan Darija text classification using LoRA (Low-Rank Adaptation). The workflow includes data preprocessing, tokenization, model training, evaluation, and performance analysis using Hugging Face Transformers and PyTorch. # 🚀 Fine-Tuning DarijaBERT for Moroccan Darija Text Classification ### 🧠 Parameter-Efficient NLP Fine-Tuning using LoRA & Hugging Face Transformers --- # 📌 Overview This project focuses on fine-tuning the **SI2M-Lab/DarijaBERT** model for **Moroccan Darija text classification** using **LoRA (Low-Rank Adaptation)**. The workflow demonstrates a complete modern NLP pipeline including: ✅ Data preprocessing ✅ Tokenization ✅ LoRA fine-tuning ✅ Model evaluation ✅ Metrics visualization ✅ Confusion matrix analysis The objective is to build an efficient NLP model capable of understanding and classifying Moroccan Darija text. --- # 🧰 Tech Stack | Category | Technologies | |---|---| | Programming | Python | | Deep Learning | PyTorch | | NLP | Hugging Face Transformers | | Fine-Tuning | LoRA / PEFT | | Data Processing | Pandas, NumPy | | Evaluation | Scikit-learn | | Visualization | Matplotlib | | Environment | Jupyter Notebook | --- # 📂 Project Structure ```bash darijabert-lora-finetuning/ │ ├── data/ │ └── dataset_darija2.csv │ ├── notebooks/ │ └── darijabert_finetuning.ipynb │ ├── results/ │ ├── confusion_matrix.png │ └── training_metrics.png │ ├── README.md ├── requirements.txt └── .gitignore ``` --- # ⚙️ Workflow ```text Dataset → Preprocessing → Tokenization → DarijaBERT → LoRA Fine-Tuning → Evaluation → Visualization ``` --- # 📊 Training Results | Metric | Score | |---|---| | 🎯 Accuracy | 79.67% | | 🧠 F1-Score | 78.91% | | 📌 Precision | 80.28% | | 🔍 Recall | 79.67% | --- # 📈 Results Visualization ## 🔥 Training Metrics --- ## 🧩 Confusion Matrix --- # 🚀 Installation Clone the repository: ```bash git clone github.com cd darijabert-lora-finetuning ``` Install dependencies: ```bash pip install -r requirements.txt ``` --- # ▶️ Run the Project Launch Jupyter Notebook: ```bash jupyter notebook ``` Open: ```bash notebooks/darijabert_finetuning.ipynb ``` --- # …