dzxpert/Taya7li.dz

THE Algerian Profanity Lingo # Taya7li.dz — Algerian Derja NSFW & Offensive Speech Lexicon A curated, bilingual dataset of offensive words, insults, and profanities in the Algerian Arabic dialect (Derja). This dataset is extracted and structured specifically to run inside lightweight semantic filtering engines to detect toxic, offensive, or NSFW texts in Algerian contexts. ## Overview Social media moderation and toxicity detection in North African dialects face unique challenges due to: 1. **Dialectal Variation**: Derja differs significantly from Modern Standard Arabic (MSA). 2. **Multilingual Scripting (Bilingualism)**: Algerian users text using both the Arabic alphabet and Latin/French scripts (commonly referred to as **Arabizi** or Chat-Arabic, incorporating numbers like `7` for `ح`, `3` for `ع`, `9` for `ق`, etc.). This project addresses these challenges by providing a structured dictionary mapping root offensive concepts to their various morphological and script spelling variations. ## Source Dataset The lexicon is extracted and curated from the **AlgD Toxicity Speech Dataset** hosted on Zenodo: * **Zenodo Record**: 10937445 * **Dataset Shape**: 14,150 comments annotated for Hate Speech, Cyberbullying, and Offensive Language. ## Extraction Methodology To construct a highly accurate and clean lexicon, the following pipeline was used: 1. **Topic-Based Filtering**: Analyzed toxicity rates across topics, highlighting categories with high concentrations of offensive speech (such as Misogyny and parliamentary politics). 2. **Statistical Token Ratio Filtering**: Surfaced candidate words by calculating the probability ratio of tokens appearing in toxic vs. clean comments: $$\text{Ratio}(w) = \frac{P(w \mid \text{Toxic})}{P(w \mid \text{Clean})}$$ 3. **Bilingual Mapping**: For every verified slang term, we mapped its standard Arabic spelling alongside its common Latin script/Arabizi phonetic spellings used in Algerian texting. 4. **False Positive Removal**: Manually removed common conversat …