mohammed-el-baraka/KhulasaAI

خلاصة · KhulasaAI Arabic Text Summarization via Knowledge Distillation --- ## Overview **KhulasaAI** is an Arabic abstractive text summarization system built using a **Knowledge Distillation** pipeline. A large teacher model (Qwen2.5-7B-Instruct) generates high-quality synthetic summaries from 5,000 Arabic Wikipedia articles, which are then used to fine-tune a compact student model (Qwen2.5-0.5B-Instruct) via **QLoRA** — achieving near-teacher performance at **14× fewer parameters**. ### Pipeline Architecture ``` ┌─────────────────────────────────────────────────────────────────────┐ │ KhulasaAI Pipeline │ ├─────────────────────────────────────────────────────────────────────┤ │ │ │ ┌──────────────┐ ┌──────────────────┐ ┌────────────────┐ │ │ │ Arabic Wiki │────▶│ Qwen2.5-7B-Inst │────▶│ Annotated │ │ │ │ 5,000 docs │ │ (Teacher Model) │ │ Dataset │ │ │ │ MSA text │ │ 4-bit quantized │ │ (text,summary)│ │ │ └──────────────┘ └──────────────────┘ └───────┬────────┘ │ │ │ │ │ ┌──────────────────────────┘ │ │ ▼ │ │ ┌──────────────────┐ ┌────────────────────┐ │ │ │ QLoRA Fine-Tune │────▶│ Student Model │ │ │ │ r=16, α=32 │ │ Qwen2.5-0.5B-Inst│ │ │ │ 1 epoch, T4 GPU │ │ 8.3MB adapter │ │ │ └──────────────────┘ └────────────────────┘ │ │ │ └─────────────────────────────────────────────────────────────────────┘ ``` --- ## Key Results Evaluated on a held-out test set of 500 samples: | Metric | Score | D …