mbuguakevvz/social-impact-pipeline

Interoperability pipeline unifying UNHCR, WHO AFRO & World Bank data for Kenya # 🇰🇪 Kenya Social Impact Interoperability Pipeline A production-grade data engineering pipeline that ingests, normalizes, and unifies social impact data from three international sources into a single queryable warehouse — exposed via a REST API and an interactive dashboard. --- ## 🎯 Problem Statement Social impact data in Kenya is fragmented across dozens of organizations — UN agencies, government bodies, NGOs, and global banks — each publishing data in different formats, schemas, and standards. This makes cross-sector analysis nearly impossible without significant manual effort. This pipeline solves that by building an **interoperability layer** that standardizes heterogeneous data into one unified schema, enabling analysts, policymakers, and humanitarian organizations to query across datasets that were previously siloed. --- ## 📦 Data Sources | Source | Data | Records | |--------|------|---------| | UNHCR Population API | Refugee, IDP & stateless populations in Kenya (2005–2023) | 61 | | WHO AFRO / GHO API | Maternal mortality, immunization, TB incidence (1985–2022) | 114 | | World Bank API | Poverty, literacy, life expectancy, GDP per capita (1992–2024) | 49 | **Total: 224 records · 12 indicators · 3 sources · 1985–2024** --- ## 🏗️ Architecture --- ## 🗂️ Project Structure --- ## 🔁 Unified Schema All sources are normalized into one table: ```sql unified_social_indicators ( record_id VARCHAR PRIMARY KEY, -- UUID source VARCHAR, -- UNHCR | WorldBank | WHO_AFRO country VARCHAR, -- ISO3 code (KEN) region VARCHAR, -- National / County indicator_name VARCHAR, -- e.g. refugee_population indicator_value DOUBLE, -- numeric value unit VARCHAR, -- percent | count | years | USD year INTEGER, -- observation year ingested_at TIMESTAMP -- pipeline run time ) ``` --- ## 🚀 Getting Star …