Amon-Mugo/Kenya-Inclusion-MPesa-Analytics

An end-to-end GCP data pipeline analyzing financial inclusion across Kenya's 47 counties and the growth of M-Pesa mobile money over a decade. Built as a portfolio project to demonstrate production-style data engineering: event-driven ingestion, dbt transformations, infrastructure as code, and a REST API — all running on Google Cloud. # Kenya Financial Inclusion & M-Pesa Analytics An end-to-end GCP data pipeline analyzing financial inclusion across Kenya's 47 counties and the growth of M-Pesa mobile money over a decade. Built as a portfolio project to demonstrate production-style data engineering: event-driven ingestion, dbt transformations, infrastructure as code, and a REST API — all running on Google Cloud. ## What this project does Three independent data sources are ingested, loaded, transformed, and served: | Source | Description | |---|---| | **World Bank Indicators** | Annual development statistics for Kenya (account ownership, bank branches, ATMs per capita), 2010–2024 | | **M-Pesa Statistics** | Yearly mobile money usage and growth metrics (agents, customers, transaction volume/value), 2015–2024 | | **FinAccess County Survey** | County-level financial inclusion metrics for all 47 Kenyan counties, 2024 FinAccess Household Survey | The pipeline ingests raw data to Cloud Storage, loads it into BigQuery, transforms it with dbt into analysis-ready marts, and serves it through a REST API and a Looker Studio dashboard. ## Architecture ``` ┌─────────────────┐ │ Data Sources │ │ World Bank API │ │ M-Pesa (static) │ │ FinAccess (static) │ └────────┬────────┘ │ ┌────────▼────────┐ │ Cloud Functions │ ◄── Ingestion (3 functions) │ (ingest) │ └────────┬────────┘ │ writes NDJSON ┌────────▼────────┐ │ GCS Bucket │ kenya-inclusion-raw │ (AFRICA-SOUTH1) │ (Eventarc finalize trigger) └────────┬────────┘ │ triggers ┌────────▼────────┐ │ Cloud Functions │ ◄── Loading (3 functions, gen2) │ (load) │ Eventarc-triggered, prefix-routed └────────┬────────┘ │ WRITE_TRUNCATE ┌────────▼────────┐ │ BigQuery │ kenya_inclusion_raw (raw tables) └────────┬────────┘ │ ┌────────▼────────┐ │ dbt │ staging → intermediate → marts │ (Cloud Run Job)│ └────────┬────────┘ │ ┌──────────────┴──────────────┐ │ │ ┌────────▼────────┐ ┌─ …