paulmanoni/tz-health-dwh

Tanzania public-health early-warning data warehouse: Google Trends + WHO/MoH + public discussion into a Kimball star schema, with outbreak anomaly detection, lead-lag correlation and a Streamlit dashboard. Python + PostgreSQL, runs offline with no API keys. # Tanzania Health Early-Warning Data Warehouse An end-to-end data warehouse that combines **Google Trends search interest**, **WHO / Ministry of Health case reports**, and **public health discussions** into a star schema, detects outbreak anomalies, measures whether online signals *lead* official case counts, and serves it all through a Streamlit dashboard. Python + PostgreSQL. Runs completely offline on committed sample data — **no API keys required**. ``` Google Trends ─┐ WHO GHO API ─┤ MoH CSV/PDF ─┼──▶ raw ──▶ staging ──▶ core (star) ──▶ marts ──▶ Streamlit Reddit / news ─┤ (typed) (Kimball) (views) HealthMap ─┘ ▲ ProMED │ anomaly detection + correlation ``` **Everything below has been run end to end**: 20/20 data-quality checks pass and 82 tests pass, including integration tests that assert the pipeline recovers the outbreaks and the 9-day reporting lag planted in the sample data. --- ## Contents - Quick start - What you get - Repository layout - The data model - CLI reference - Airflow - Going live - Testing - Troubleshooting - Documentation --- ## Quick start ### Prerequisites - **PostgreSQL 15+**, already running. This project targets a local install (Postgres is deliberately *not* containerised) and defaults to **port 5472**. - **Python 3.11+** (developed and tested on 3.14). - Docker Desktop — optional, only for Airflow. ### 1. Install ```bash cd disease python3 -m venv .venv .venv/bin/pip install -r requirements.txt ``` ### 2. Configure ```bash cp .env.example .env ``` Then set `DB_PASSWORD` in `.env` to your PostgreSQL password. The other defaults match a local install on **port 5472** with user `postgres` — edit them if yours differ. `.env` is git-ignored; never put a real password in `.env.example`. `OFFLINE_MODE=true` is the default and is what makes the project run with no keys and no network. ### 3. Create the warehouse and run the pipeline …