An end-to-end data engineering and machine learning pipeline for forecasting Nigeria's retail petrol prices, detecting structural market breaks, and generating actionable intelligence for policymakers and MSMEs
# 🇳🇬 Nigeria Retail Fuel Price Intelligence System
> **Uni Dissertation Project — Distinction Track**
> An end-to-end data engineering and machine learning pipeline for forecasting Nigeria's retail petrol prices, detecting structural market breaks, and generating actionable intelligence for policymakers and MSMEs.
---
---
## Table of Contents
- Project Overview
- Research Questions
- Dataset
- Architecture
- Kernel Map
- Key Insights Implemented
- Tech Stack
- Project Structure
- Getting Started
- Data Sources
- Outputs
- Model Architecture
- Results
- Policy Applications
- Thesis Citation
- References
- License
---
## Project Overview
Nigeria's retail fuel market underwent a historic structural transformation in May 2023 when the incoming Tinubu administration removed decades-old petroleum subsidies. This event — combined with the Dangote Refinery coming online in 2024 — shifted the market from a state-controlled import monopoly (NNPCL) to an **asymmetric duopoly** with volatile, market-reflective pricing.
This project builds a **production-grade forecasting and intelligence system** that:
- Engineers a clean analytical data warehouse from raw WFP price data using **SQL (DuckDB)**
- Detects structural market breaks using the **PELT change-point detection algorithm**
- Trains a **2-layer multivariate LSTM** with a custom trust-weighted loss function
- Generates a **3-month early-warning fuel price forecast** to help MSMEs hedge logistics costs
- Maps **regional price disparities** to support targeted government cash-transfer programmes
- Produces **fully interactive Plotly visualisations** for all analytical outputs
---
## Research Questions
1. **Duopoly Shift** — Can data distinguish price hikes caused by global crude shocks (external) from domestic refinery bottlenecks (internal)?
2. **MSME Vulnerability** — Which months trigger a "volatility squeeze" where MoM price surges erode MSME profit margins?
3. **Regional Disparity** — Which Nigerian s …