Abdeljalil-Ounaceur/FineTuning-NLLB-200-For-Tamazight-Souss
Domain:
natural language processingRecord type:
model
Creator:
Abd
Host:
# Fine-tuned NLLB-200 for English-Tamazight Translation
This project fine-tunes the NLLB-200 (No Language Left Behind) model for English-Tamazight translation using a comprehensive dictionary dataset.
## 📊 Dataset
- **Source**: Abdeljalil-Ounaceur/English-Tamazight-Dictionnary-2007
- **Size**: ~9,000+ translation pairs
- **Content**:
- Dictionary entries (English ↔ Tamazight)
- Verb conjugations
- Country names
- Cultural/religious phrases
- **Target Language**: `kab_Latn` (Kabyle Latin script)
## 🚀 Fine-tuning Process
### Base Model
- **Model**: `facebook/nllb-200-distilled-600M`
- **Architecture**: Sequence-to-sequence transformer
- **Original Training**: 200 languages with billions of sentence pairs
### Training Configuration
- **Epochs**: 3
- **Learning Rate**: 3e-5
- **Batch Size**: 8 (per device)
- **Max Length**: 128 tokens
- **Evaluation**: BLEU score on validation set
- **Hardware**: GPU-accelerated training
### Data Preprocessing
1. Combined all TSV files from the dataset
2. Created bidirectional translation pairs (EN→ZGH and ZGH→EN)
3. 90/10 train/validation split
4. Tokenized using NLLB tokenizer with proper language codes
## 📁 Files
- `English_tamazight_NLLB_FineTuning.ipynb`: Complete Jupyter notebook with training pipeline
- `english_tamazight_nllb_finetuning.py`: Python script version of the training code
## 🔧 Usage
### Load the Fine-tuned Model
```python
from transformers import AutoTokenizer, AutoModelForSeq2SeqLM
# Load from Hugging Face Hub
model_name = "Abdeljalil-Ounaceur/nllb-tamazight-souss"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForSeq2SeqLM.from_pretrained(model_name)
```
### Compare with Original NLLB
```python
# Compare normal NLLB vs fine-tuned model
from transformers import AutoTokenizer, AutoModelForSeq2SeqLM
import torch
# Determine the device
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
print(f"Using device: {device}")
# Load both models
print("Loading models.. …