jefftrojan/jefftrojan-hausa-english-codeswitching

Language Identification and Sentiment Analysis in Code-Switched Hausa-English Text: A Study of West African Social Media Pragmatics # HausaCS — Hausa-English Code-Switching Dataset & Benchmarks > **Language Identification and Sentiment Analysis in Code-Switched Hausa-English Text: > A Study of West African Social Media Pragmatics** --- ## Overview HausaCS is a manually annotated corpus of ~5 000 code-switched Hausa-English utterances collected from Twitter/X, Facebook, and WhatsApp broadcasts in northern Nigeria (2022–2024). Each token is tagged for **language identity** (HAU / ENG / MIXED / OTHER) and each utterance for **sentiment** (POS / NEG / NEU). The repository contains: | Path | Contents | |------|----------| | `experiments/` | Fine-tuning & evaluation scripts (mBERT, XLM-R, AfriBERTa) | | `figures/` | Matplotlib-generated paper figures (PNG) | | `docs/` | HuggingFace dataset card | | `build_annotation_sheet.py` | Generates `HausaCS_annotation_template.xlsx` | | `generate_figures.py` | Generates all paper figures | | `generate_paper.py` | Generates baseline paper draft (.docx) | | `generate_jom_paper.py` | Generates Journal of Metaverse-formatted paper (.docx) | | `README_dataset.md` | HuggingFace dataset card (YAML + Markdown) | --- ## Experiment Scripts Run in order: ```bash # 1. Baseline fine-tuning (mBERT, XLM-R, AfriBERTa) python experiments/01_baseline_benchmark.py # 2. LID-prefix ablation (gold vs. predicted prefix) python experiments/02_lid_prefix.py # 3. Error analysis python experiments/03_error_analysis.py # 4. Inter-annotator agreement (Cohen's κ) python experiments/04_kappa.py # 5. Prepare & upload HuggingFace dataset python experiments/05_prepare_hf_dataset.py ``` ### Requirements ``` transformers>=4.38 datasets>=2.18 torch>=2.1 scikit-learn>=1.4 openpyxl>=3.1 matplotlib>=3.8 python-docx>=1.1 ``` Install with: ```bash pip install -r requirements.txt ``` --- ## Annotation Open `HausaCS_annotation_template.xlsx` (generated by `build_annotation_sheet.py`) in Excel or LibreOffice Calc. The workbook contains: - **Instructions** — annotation guidelines wit …