Hedi-Bk/tunisia-law-finetuned-llms

Fine-tuned Gemma 4 E4B and Qwen3 8B models on Kaggle for Tunisian Law using Unsloth. # 🇹🇳 Tunisian Law Fine-Tuned LLMs This repository contains links and resources for fine-tuned Large Language Models specialized in Tunisian Law. The models were trained using Unsloth on Kaggle notebooks. --- # Project Goal The objective of this project is to develop domain-specific Small Language Models (SLMs) specialized in Tunisian Law through QLoRA fine-tuning. The models were trained on a synthetic legal instruction dataset generated using a teacher LLM. The dataset contains approximately 1,660 high-quality legal conversation examples grounded in real Tunisian legal texts and contextual legal documents. The training data follows a conversational instruction-tuning format composed of: - System prompts defining the legal assistant behavior - User legal questions with supporting legal context/documents - Structured assistant answers citing legal articles and sources The models were fine-tuned to perform legal reasoning and legal question answering based strictly on provided legal context. # Main Objectives The goal of these Tunisian Law SLMs is to enable: - Legal question answering grounded in legal documents - Tunisian legal document understanding - Arabic/French/Derja legal reasoning - Efficient local inference using GGUF - Lightweight legal AI systems deployable on consumer hardware --- # Dataset Characteristics The dataset contains multilingual Tunisian legal conversations in: - Arabic - Tunisian Arabic (Derja) - French Each example is designed to simulate realistic legal assistant interactions for the E-Tafakna legal platform. Example capabilities include: - Context-aware legal question answering - Article and source citation - Legal document understanding - Multilingual legal reasoning - Retrieval-style grounded responses - Structured legal explanations --- # Models ## 🔹 Qwen3 8B Models Fine-tuned versions of the Qwen3 8B model on Tunisian Law datasets. Two training configurations are available: - **2 epochs** - **3 epochs** --- ### …