Abdennour03/Darija-LLM-Finetuning

# XDarija: Fine-Tuning LLMs on Darija & Tamazight This repository is dedicated to fine-tuning large language models (LLMs) on **Darija** (Moroccan Arabic) and **Tamazight** (Tifinagh) datasets. Our mission is to identify the most efficient methods and optimal hyperparameters to build models that accurately understand and generate Moroccan dialects and languages. ## 🔗 Project Links * **GitHub Repository:** Abdennour03/Darija-LLM-Finetuning * **Hugging Face Hub:** Open-Darija-Tamazight Organization --- ## 🎯 Project Objectives * **Data Excellence:** Prepare and clean high-quality Darija and Tamazight textual data. * **State-of-the-Art Training:** Apply modern fine-tuning techniques (LoRA, QLoRA, PEFT). * **Optimization:** Systematically test architectures and hyperparameters to find the "sweet spot" for Moroccan NLP. * **Open Science:** Share models, code, and findings with the community to accelerate research in North African languages. --- ## 🚀 Model Roadmap ### 1. Chat Darija * **Focus:** Conversational Moroccan Arabic. * **Technique:** Fine-tuned using **PEFT (Parameter-Efficient Fine-Tuning)** and **LoRA**. * **Goal:** Natural interaction in the local dialect. ### 2. Tamazight Translation * **Base Model:** `nllb-200-distilled-600M` * **Focus:** Accurate translation between Tamazight (Tifinagh script) and other languages. * **Technique:** Domain-specific fine-tuning for low-resource linguistic accuracy. --- ## 🛠️ Getting Started To contribute or replicate the fine-tuning process: 1. Clone the repo: `git clone github.com` 2. Explore the training notebooks in the `model-translation/` directory. 3. Check our latest weights on Hugging Face.