DRAVIS55/africa-news-nlp-pilot

NLP pilot for news analysis, prediction, and relational intelligence focused on the African news ecosystem. Kenya-first, continent-ready. # News Headline AI — Fine-Tuned Language Model ## What This Is This project is a **pilot proof-of-concept** for a larger vision: an AI system that understands African news, detects patterns across time, and assists journalists with research and writing. This pilot specifically trains a small language model to do **one focused task**: > Given a full news article, generate a concise, accurate headline. It is the foundation of something bigger. Once you understand how fine-tuning works here — swapping the dataset, scaling the model, and adding new task prefixes — you can extend this into: - Historical pattern matching ("when did Kenya last see this?") - Peer country comparison briefings - Swahili / Sheng article summarisation - Political trend detection across East Africa This pilot runs entirely on your laptop. No API. No internet after setup. No ongoing cost. --- ## How It Works The model used is **T5-small** — a 60MB open-source text-to-text transformer originally built by Google and released publicly. It is not ChatGPT. It is small enough to train on a CPU laptop in under an hour. Fine-tuning means we take this pre-trained model (which already understands English grammar and structure) and continue training it specifically on news article → headline pairs. After training, the model has learned the pattern of how good headlines are written from article content. The training data comes from **XSum** — a dataset of 226,000 BBC news articles each paired with a human-written one-sentence summary. We use 2,000 articles for training and 200 for validation so it finishes in reasonable time on a CPU. ``` Input → "The Central Bank of Kenya has raised the base lending rate by 50 basis points citing persistent inflationary pressure..." Output → "Kenya central bank raises lending rate amid inflation pressure" ``` The trained model is saved to your machine and runs fully offline from that point forward. --- ## Project Structure After cloning and running, your …