Logo Lanfrica

Modeling North African Dialects from Standard Languages

Domaine:

natural language processing

Type de record:

papermodel
Créateur:
TouSmaLan
Éditeur:
StaLabANR
Éditeur:
CCSD
Hôte:avatar
International audience Processing North African Arabic dialects presents significant challenges due to high lexical variability, frequent code-switching with French, and the use of both Arabic and Latin scripts. We address this with a phonemebased normalization strategy (Toughrai et al., 2025) that maps Arabic and French text into a simplified representation (Arabic rendered in Latin script), reflecting native reading patterns. Using this method, we pretrain BERTbased models on normalized Modern Standard Arabic and French only and evaluate them on Named Entity Recognition (NER) and text classification. Experiments show that normalized standard-language corpora yield competitive performance on North African dialect tasks; in zero-shot NER, Ar_20k surpasses dialectpretrained baselines. Normalization improves vocabulary alignment, indicating that normalized standard corpora can suffice for developing dialect-supportive language models in lowresource contexts.