sogodongo/angaza

Real-time AQI intelligence for 10 African cities.Kinesis, Flink, Lambda, Glue, and SNS alerts. Built in Nairobi. # Angaza > *Angaza* - Swahili for "to illuminate, to shed light" Air quality across African cities is poorly monitored, unevenly reported, and almost never available in real time. Angaza changes that. It is a production-grade streaming and batch analytics platform that ingests AQI sensor data from 10 African cities, processes it through a fully managed AWS pipeline, and fires targeted alerts within seconds of a hazardous threshold breach - all from a single `python3 infra/deploy.py`. Built in Nairobi. Engineered against the infrastructure constraints most tutorials never mention. --- ## How it works Data moves in two paths simultaneously. The **real-time path** runs continuously: a Python producer puts AQI readings onto a Kinesis stream every second, Kinesis Data Analytics applies 5-minute tumbling window aggregations via Apache Flink, and Lambda picks up the enriched records - publishing CloudWatch metrics per city and triggering SNS alerts the moment any city crosses AQI 150. The **batch path** runs in parallel: Kinesis Firehose buffers the same raw records and delivers them to S3 every 60 seconds, partitioned by date. A Glue crawler runs every 15 minutes, keeping the Athena catalog current so analysts can query the full historical dataset without touching the live stream. --- ## Stack | Layer | Technology | Why | |---|---|---| | Ingestion | Kinesis Data Streams - 2 shards | Managed, native Firehose + Lambda integration | | Batch delivery | Kinesis Firehose → S3 | Zero-ops delivery, dynamic date partitioning | | Stream analytics | Kinesis Data Analytics (Flink) | Windowed aggregations without managing clusters | | Serverless compute | AWS Lambda - Python 3.11 | Per-city metric publishing and alert fanout | | Storage | S3 - raw / analytical / checkpoints | Medallion layout, versioning, lifecycle tiering | | Catalog | AWS Glue + Athena | Schema-on-read, no ETL pipeline to maintain | | Alerting | SNS - email + SQS | Decoupled, extensible subscriber …