Souadzriouil/darijabert-lora-finetuning
Domaine:
natural language processingType de record:
project
Créateur:
Sou
Hôte:
This project focuses on fine-tuning DarijaBERT for Moroccan Darija text classification using LoRA (Low-Rank Adaptation). The workflow includes data preprocessing, tokenization, model training, evaluation, and performance analysis using Hugging Face Transformers and PyTorch.
# 🚀 Fine-Tuning DarijaBERT for Moroccan Darija Text Classification
### 🧠 Parameter-Efficient NLP Fine-Tuning using LoRA & Hugging Face Transformers
---
# 📌 Overview
This project focuses on fine-tuning the **SI2M-Lab/DarijaBERT** model for **Moroccan Darija text classification** using **LoRA (Low-Rank Adaptation)**.
The workflow demonstrates a complete modern NLP pipeline including:
✅ Data preprocessing
✅ Tokenization
✅ LoRA fine-tuning
✅ Model evaluation
✅ Metrics visualization
✅ Confusion matrix analysis
The objective is to build an efficient NLP model capable of understanding and classifying Moroccan Darija text.
---
# 🧰 Tech Stack
| Category | Technologies |
|---|---|
| Programming | Python |
| Deep Learning | PyTorch |
| NLP | Hugging Face Transformers |
| Fine-Tuning | LoRA / PEFT |
| Data Processing | Pandas, NumPy |
| Evaluation | Scikit-learn |
| Visualization | Matplotlib |
| Environment | Jupyter Notebook |
---
# 📂 Project Structure
```bash
darijabert-lora-finetuning/
│
├── data/
│ └── dataset_darija2.csv
│
├── notebooks/
│ └── darijabert_finetuning.ipynb
│
├── results/
│ ├── confusion_matrix.png
│ └── training_metrics.png
│
├── README.md
├── requirements.txt
└── .gitignore
```
---
# ⚙️ Workflow
```text
Dataset → Preprocessing → Tokenization → DarijaBERT → LoRA Fine-Tuning → Evaluation → Visualization
```
---
# 📊 Training Results
| Metric | Score |
|---|---|
| 🎯 Accuracy | 79.67% |
| 🧠 F1-Score | 78.91% |
| 📌 Precision | 80.28% |
| 🔍 Recall | 79.67% |
---
# 📈 Results Visualization
## 🔥 Training Metrics
---
## 🧩 Confusion Matrix
---
# 🚀 Installation
Clone the repository:
```bash
git clone github.com
cd darijabert-lora-finetuning
```
Install dependencies:
```bash
pip install -r requirements.txt
```
---
# ▶️ Run the Project
Launch Jupyter Notebook:
```bash
jupyter notebook
```
Open:
```bash
notebooks/darijabert_finetuning.ipynb
```
---
# …