SalimaYoula/diffey-health-platform

Data engineering platform for maternal health analytics in West Africa — supporting the Diffey project # Diffey Health Data Platform > *"Devenir mère, un don du ciel"* A data engineering platform analyzing maternal and child health indicators across West Africa, built to support the **Diffey project** — a community initiative helping women navigate their pregnancy journey safely. ## The Problem Every day, women in West Africa face a reality that should not exist in 2024: | Country | Maternal Mortality (per 100k births) | vs France | |---------|--------------------------------------|-----------| | Guinea | 494 | 71x more risk | | Nigeria | 1,001 | 143x more risk | | Mali | 354 | 51x more risk | | France | 7 | baseline | This platform transforms raw World Bank data into actionable insights to support decision-makers, NGOs, and health professionals. ## Architecture ```mermaid flowchart LR A[World Bank Open Data API] -->|extract.py requests + pandas| B[(Raw layer Parquet)] B -->|dbt staging stg_health| C[Staging model] C -->|dbt marts 5 models + tests| D[(DuckDB analytical marts)] D --> E[Streamlit dashboard Plotly maps & charts] ``` **Flow:** 1. **Extract** — `extract.py` pulls 5 health indicators for 12 countries (2000–2023) from the World Bank Open Data API and lands them as Parquet 2. **Transform** — dbt cleans and standardises the raw data in a staging model (`stg_health`), then builds 5 analytical marts (one per indicator), with schema tests and a custom data-quality test on maternal mortality ranges 3. **Serve** — the DuckDB marts power a Streamlit dashboard with Plotly maps and cross-country comparisons ## Tech Stack | Tool | Purpose | |------|---------| | Python | Data extraction and pipeline orchestration | | requests | World Bank API calls | | dbt | SQL transformations, testing, documentation | | DuckDB | Analytical storage | | Streamlit | Interactive dashboard | | Plotly | Data visualization and maps | | Parquet …