ismailSadouki/dziri-dpo
Domain:
natural language processingRecord type:
softwareproject
Creator:
ism
Host:
End-to-end DPO alignment pipeline from scratch. Implements DPO mechanics in pure PyTorch with numerical verification against TRL, followed by SFT and DPO training, preference data construction, annotation agreement, and evaluation for Algerian Darija alignment.
# DziriDPO
**End-to-end DPO alignment pipeline from scratch.** Implements DPO mechanics in pure PyTorch with numerical verification against TRL, followed by SFT and DPO training, preference data construction, annotation agreement, and evaluation for Algerian Darija alignment.
> **Public framing:** verified DPO mechanics + Darija alignment methodology — not simply “I fine-tuned a model.”
---
## Overview
DziriDPO is a two-stage project for understanding and building Direct Preference Optimization (DPO) systems.
The project separates **DPO mechanics verification** from **production Darija alignment**:
```text
DziriDPO
│
┌────────────┴────────────┐
│ │
▼ ▼
Scratch DPO Darija Alignment
Pure PyTorch TRL
│ │
▼ ▼
DPO mechanics Data + SFT + DPO
│ │
▼ ▼
TRL numerical oracle Triple-axis evaluation
│ │
└────────────┬────────────┘
▼
Reproducible results
```
The central correctness oracle is:
$$
|L_{\text{scratch}} - L_{\text{TRL}}| **What makes the chosen response better than the rejected response in Algerian Darija?**
The guideline considers properties such as:
* instruction following
* factual correctness
* relevance
* clarity
* natural Darija usage
* appropriate code-switching
* cultural/contextual appropriateness
* harmful or misleading content
The detailed guideline is maintained in:
```text
darija_alignment/data/guideline.md
```
---
# Inter-Annotator Agreement
A subset of at least **100 preference pairs** is independently double-annotated.
Cohen's kappa is computed as:
$$
\kappa
======
\frac{p_o-p_e}{1-p_e}
$$
where:
* $p_o$ is observed agreement
* $p_e$ is expected agreement by chance
The report includes:
* number of double-annotated pairs
* observed agreement
* expected agreement
* Cohen's $\kappa$
* disagreement categories
* re …