sarahammoudaa/Qlora-tunisian-dialect-unsloth

# Tunisian Arabic (Darija) Task Dataset Fine-Tuning with Gemma 2 and Unsloth 🦥 ## Overview This repository demonstrates fine-tuning of large language models (LLMs) on **Tunisian Arabic (Darija)** using the **Gemma 2** model from Google and **Unsloth** for fast, memory-efficient training. The goal is to enable high-quality text generation, question answering, and language understanding in Darija using a lightweight yet powerful LLM setup. --- ## Dataset: Darija Task Dataset The **Darija Task Dataset** is a curated collection designed for LLM fine-tuning, focusing specifically on Tunisian and Moroccan Arabic (Darija). ### Key Features: - Contains **parallel translations and linguistic tasks** across Darija, Arabic, and English. - Streamlined for **easy fine-tuning** in a single CSV file (`TuniziDataset.csv`). - Includes tasks such as: - Question answering - Text completion - Translation between Darija and Arabic ### Sample Data: | Index | Input Text | Sentiment Label | |-------|------------|----------------| | 52693 | ekbess yar7em bouk | 1 | | 52694 | rabi m3ak ya mr moncef 7atta ken marba7tech fi el2inti5abet enti rba7tna a7na | 1 | | 52711 | ti yezi mil kethb w nifa9 li matou fi sousa 3bad w li matou b train chnouma hathoukom | -1 | --- ## Tools & Models Used ### **Unsloth 🦥** Unsloth enables **2x faster fine-tuning** of LLMs with **70% less VRAM**. It supports: - OpenAI GPT-OSS - DeepSeek-R1 - Qwen3 - Gemma 3 - Gemma 2 - TTS training **Website / Docs:** Unsloth GitHub ### **Gemma 2** Gemma is a **lightweight, state-of-the-art open model family** by Google, built with the same technology used for Gemini models. **Features:** - Decoder-only, text-to-text LLMs - Instruction-tuned and pre-trained variants - Ideal for question answering, summarization, reasoning - Small model size enables deployment on **laptops, desktops, or personal cloud servers** **Resources:** - Gemma 2 Model Card - Gemma on Kaggle - Gemma Vertex Model Garden --- ## Project Structure …