Fadene-Akram/Franco2Fosha

Franco2Fosha is a comprehensive end-to-end neural NLP pipeline for direct translation of : Arabizi → Algerian Darija (Arabic script - dialectal form), Arabizi → Modern Standard Arabic (MSA) (formal Arabic - standardized) # Franco2Fosha: Neural Translation of Algerian Arabizi to Darija and Modern Standard Arabic | | | |:---:|:---:| | | | | _Arabizi Transliteration_ | _Darija Transliteration_ | | | | | _MSA Transliteration_ | _Corpus Contribution_ | ## 📖 Overview **Franco2Fosha** is a comprehensive end-to-end neural NLP pipeline for **direct translation** of **Algerian Arabizi** (Latin-script Arabic dialect) to two output targets: - **Arabizi → Algerian Darija** (Arabic script - dialectal form) - **Arabizi → Modern Standard Arabic (MSA)** (formal Arabic - standardized) Users can translate to either Darija (to preserve dialectal form) or MSA (to standardize to formal Arabic) in a single inference step. This project bridges the critical gap between informal, user-generated digital content and formal linguistic resources for low-resource North African Arabic dialects. ### The Problem: Arabizi as "Dark Matter" for NLP **Arabizi** is a spontaneous orthography that uses Latin characters and numerals (arithmograms) to represent Arabic dialects online. While it enables rapid communication for millions of Algerians, it creates fundamental challenges for NLP systems: - **Orthographic Inconsistency**: The word "why" can be written as `3lah`, `3leh`, `pk`, `pourquoi`, or `3lach` (spelling variance problem) - **Resource Scarcity**: Unlike Egyptian or Levantine dialects, Algerian lacks large-scale parallel corpora (low-resource NLP challenge) - **Script Mismatch**: Standard NLP tools trained on Arabic script fail catastrophically on Romanized text (domain shift problem) - **Code-Switching Intensity**: 35% of sentences contain French loanwords, requiring multilingual understanding ### Our Solution: Parallel Translation Models Franco2Fosha addresses this challenge using **two parallel neural architectures**, each specialized for its target: 1. **ByT5-Small Model**: Arabizi → Algerian Darija (character-level transliteration) 2. **mBART-Large-50 Model**: Arabizi → Modern Stand …