Mackey-Kumi/ghana-clickbait-detection

Detecting clickbait and sensationalism in Ghanaian online news headlines -- a research comparison of TF-IDF+LR, fine-tuned DistilBERT, and zero-shot LLM classification on a purpose-built, human-annotated dataset. # Detecting Clickbait and Sensationalism in Ghanaian Online News Headlines A research project comparing three modeling approaches — classical ML, a fine-tuned transformer, and a zero-shot LLM — on a purpose-built, human-annotated dataset of Ghanaian news headlines. DCIT 316 — Computational Models for Social Media Mining, University of Ghana, 2025/2026. Scoped as a research comparison, not a deployed product: the deliverable is the dataset, the annotation methodology, and the model comparison, not a web app. ## Results at a glance Same 1,000-headline dataset, same seed=42 80/20 test split, all three models: | | Accuracy | Macro F1 | Clickbait Precision | Clickbait Recall | |---|---|---|---|---| | Baseline (TF-IDF + Logistic Regression) | 0.737 | 0.647 | 0.390 | 0.590 | | **Fine-tuned DistilBERT** | **0.783** | **0.714** | **0.468** | **0.744** | | Zero-shot (`bart-large-mnli`, literal labels) | 0.798* | 0.444 | 0.000 | 0.000 | | Zero-shot (`bart-large-mnli`, descriptive labels) | 0.788 | 0.651 | 0.457 | 0.410 | \* Zero-shot's raw accuracy is a majority-class artifact on this imbalanced dataset, not evidence of real classification skill — see `notebooks/zeroshot.ipynb`. Fine-tuned DistilBERT wins outright, and also generalizes best to headlines from outlets it never trained on (held-out-outlet Clickbait recall 0.744 vs. the baseline's 0.453). Full methodology and discussion in each stage's notebook, linked below. ## Setup ```bash pip install -r requirements.txt ``` ## Stage 1 — Scraping ```bash python -m scraper.scrape --dry-run # fetch ~10 headlines/outlet, sanity-check selectors python -m scraper.scrape # full collection run, ~1,000 headlines ``` (Run as a module — `python scraper/scrape.py` directly fails with `ModuleNotFoundError: No module named 'scraper'`, since the script's own directory, not the project root, ends up on `sys.path`.) Output: `data/raw/ .csv` per outlet, plus `data/raw/headlines_combined.csv`. Full source list, collect …