slimzrrk/Tunisian-Dialect-Model

# Abajim - Benchmark-Driven Fine-Tuning for Tunisian Arabic Educational Assistants ## Abstract Tunisian primary-school children learn in Modern Standard Arabic at school, but they ask for help at home in Tunisian dialect and often in Arabizi. This creates a concrete educational gap: existing Arabic assistants can generate fluent text, yet they frequently miss the local register, the script used by children, or the pedagogical structure required for homework support. Abajim addresses this gap through a fully local, benchmark-driven pipeline for dialect-aware educational assistants. The project combines LoRA fine-tuning of Qwen2.5-7B-Instruct with MLX on Apple Silicon, a structured educational SFT dataset, and a task-oriented benchmark covering script, task type, and pedagogical quality. On the official 84-example benchmark, `qwen25-edu-v1-fused` reaches `3.1544` versus `2.7277` for the Labess baseline and wins on `67/84` examples. On a corrected-generation diagnostic subset, the same model reaches `4.021` in pedagogy level and `2.465` in explanation quality, showing that generation calibration matters, even though Arabizi off-topic behavior remains an open problem. ## Original Contributions - A structured SFT dataset for Tunisian primary-school assistance, with explicit task types such as correction, exercise, explanation, quiz, rag_answer, and dialogue. - A reproducible benchmark protocol for dialectal Arabic and Arabizi educational assistants, scored on language adherence, pedagogy, explanation quality, subject adherence, and output health. - A complete local LoRA fine-tuning pipeline on Apple Silicon with MLX, documented from dataset preparation to benchmarkable fused checkpoints. - A diagnostic methodology that separates generation artefacts from real behavioral failures, preventing incorrect conclusions during model iteration. - A model-specific inference profile, `qwen_standard`, showing that dialect evaluation quality depends not only on training data but …