mohammed-el-baraka/KhulasaAI
Domain:
natural language processingRecord type:
model
Creator:
Moh
Host:
خلاصة · KhulasaAI
Arabic Text Summarization via Knowledge Distillation
---
## Overview
**KhulasaAI** is an Arabic abstractive text summarization system built using a **Knowledge Distillation** pipeline. A large teacher model (Qwen2.5-7B-Instruct) generates high-quality synthetic summaries from 5,000 Arabic Wikipedia articles, which are then used to fine-tune a compact student model (Qwen2.5-0.5B-Instruct) via **QLoRA** — achieving near-teacher performance at **14× fewer parameters**.
### Pipeline Architecture
```
┌─────────────────────────────────────────────────────────────────────┐
│ KhulasaAI Pipeline │
├─────────────────────────────────────────────────────────────────────┤
│ │
│ ┌──────────────┐ ┌──────────────────┐ ┌────────────────┐ │
│ │ Arabic Wiki │────▶│ Qwen2.5-7B-Inst │────▶│ Annotated │ │
│ │ 5,000 docs │ │ (Teacher Model) │ │ Dataset │ │
│ │ MSA text │ │ 4-bit quantized │ │ (text,summary)│ │
│ └──────────────┘ └──────────────────┘ └───────┬────────┘ │
│ │ │
│ ┌──────────────────────────┘ │
│ ▼ │
│ ┌──────────────────┐ ┌────────────────────┐ │
│ │ QLoRA Fine-Tune │────▶│ Student Model │ │
│ │ r=16, α=32 │ │ Qwen2.5-0.5B-Inst│ │
│ │ 1 epoch, T4 GPU │ │ 8.3MB adapter │ │
│ └──────────────────┘ └────────────────────┘ │
│ │
└─────────────────────────────────────────────────────────────────────┘
```
---
## Key Results
Evaluated on a held-out test set of 500 samples:
| Metric | Score | D …