YXlh-64/Algerian-Dialect-Diacritization

# Algerian Dialect Diacritization — AISI Research Project ## Overview Diacritization is the task of restoring missing short vowels and other diacritic marks in Arabic text, which is critical for improving readability, pronunciation support, and downstream Arabic NLP tasks such as text-to-speech, speech recognition, and linguistic analysis. Algerian dialect Arabic (Derja) is particularly challenging because of its high orthographic variation, code-switching tendencies, limited standardized writing conventions, and scarcity of publicly available diacritized resources. This project targets two core contributions: (1) developing the first structured Algerian dialect diacritized dataset for research use, and (2) establishing transfer learning baselines across multiple pretraining regimes and model families. ## Experimental Strategies Every run in this repo also belongs to one of four transfer-learning strategies (independent of which track/architecture it uses): | Strategy | Pretraining Data | Fine-tuning Data | Research Question | |----------|------------------|------------------|-------------------| | A | None | Algerian only | Low-resource baseline — how hard is the task? | | B | MSA (Tashkeela ~50k) | Algerian | Does standard Arabic help? | | C | NADI (~15k) | Algerian | Does multi-dialect Arabic help? | | D | MSA + NADI (~65k) | Algerian | Does combined data help? | **All results currently in this repo (`experiments/leaderboard.md` and every `configs/**/strategy_a_*.yaml`) are Strategy A** — no external pretraining, char/token classifier trained (or fine-tuned) directly on the Algerian data. Strategies B/C/D are on the roadmap but not yet implemented. ## The Four Tracks The project compares four families of architecture for the same char-level diacritic-tagging task. Each track is a top-level folder under `configs/`, `models/`, `training/`, `evaluation/`, and `experiments/` (e.g. `training/track3/...`). | Track | Name | Architectures | Status | |---|---|- …