# π³π¬ Naija News Hub
A live, full-stack Nigerian news aggregator β built end-to-end with a scheduled ETL pipeline, an ML-powered categorization layer, a FastAPI backend, and a modern Next.js frontend with infinite scroll, search, and a dark/light theme.
Naija News Hub pulls headlines from five major Nigerian publishers, cleans and enriches them (real photos, article categories, view-based popularity), and serves them through a searchable, filterable web app that refreshes every 10 minutes β all without ever rewriting or republishing full article content. Every story links straight back to its original publisher.
---
## β¨ Features
- **Live aggregation** from 5 sources β Punch, Vanguard, Premium Times, Daily Post, and PM News β refreshed every 10 minutes via Apache Airflow
- **Automatic deduplication** β the same story is never inserted twice, even across repeated runs
- **Real per-article images** β extracted from each publisher's actual article page (`og:image` / `twitter:image` meta tags), not just generic feed thumbnails
- **ML-based topic categorization** β a zero-shot NLI classifier (`valhalla/distilbart-mnli-12-1`) tags every article into Politics, Sports, Business, Entertainment, Technology, Health, Crime, or World with no manual labeling required
- **Popularity tracking** β view counts recorded on click-through, surfaced in a homepage carousel of the most-read stories
- **Full-text search & multi-dimensional filtering** β by keyword, source, category, and date (year β month β day)
- **Archive browsing** β explore any day, month, or year of coverage
- **Resilient scraping** β automatic fallback through direct requests β residential proxy (Oxylabs) β graceful skip, so one blocked source never takes down the whole pipeline
- **Dark / light theme toggle** with a custom editorial design system (glassmorphism, animated live indicator, no generic "AI dark mode" look)
- **Memory-optimized infinite scroll** β windowed rendering (TanStack Virtual) keeps the DOM lig β¦