layhadarries/Cape-Town-Suburb-Energy-Vulnerability-Index

A data pipeline to extract dam level and electricity/ loadshedding data from EskomSePush and Cape Town Open Data API. Output is a Google Data Studio report, providing insight into the current loadshedding schedule and dam levels across South Africa. # Cape Town Suburb Energy Vulnerability Index A data engineering pipeline that scores Cape Town suburbs by how exposed they are to load shedding — combining actual outage hours with property value, commercial/zoning density, and public facility density — to answer a genuinely useful question: **which suburbs get hit hardest by load shedding, relative to what's actually at stake there?** Two suburbs can have identical loadshedding stages and hours, but if one has three clinics and a commercially-zoned business strip and the other is mostly low-density residential, the real-world impact is completely different. This project builds a composite index that captures that difference, not just raw outage hours. ## The question this answers > For each suburb: how many hours of load shedding does it actually experience, and how much economic/civic activity sits behind that suburb (property value, commercial zoning density, public facilities)? Combine both into a single comparable score. ## Architecture ``` ┌─────────────────────┐ ┌────────────────────────────┐ │ EskomSePush API │ │ CT Open Data Portal │ │ (outage hours per │ │ (property valuations, │ │ tracked suburb) │ │ zoning, civic facilities)│ └──────────┬──────────┘ └─────────────┬──────────────┘ │ │ └───────────────┬────────────┘ ▼ AWS S3 (raw landing zone) │ ▼ AWS RDS Postgres (warehouse) dim_suburb + fact tables │ ▼ dbt (build vulnerability index mart, normalize, weight, test) │ ▼ Looker Studio (suburb map + ranked score table) All of the above (except Looker Studio) is orchestrated by Apache Airflow, running in Docker. ``` ## Tech stack | Layer | Tool | |---|---| | Language | Python 3 | | Extraction | `requests` (CT Open Data), custom EskomSePush client | | Raw storage | AWS S3 | | Warehouse | AWS RDS (Postgres) | | Transformation | dbt-core (`dbt-postgres`) | | Orchestration | Apache Airflow (Docker Compose) | | Visualization | Looker Studio | | …