ChachaMarwaDev/kenya-climate-risk-monitor
Domaine:
climateType de record:
softwareproject
Créateur:
Cha
Hôte:
GCP pipeline for real-time drought and flood risk monitoring across Kenyan counties using open weather APIs, BigQuery, and Looker Studio.
# Kenya Climate Risk Monitor
### Drought & Flood Early Warning System
## Problem Statement
Kenya's 47 counties face recurring drought and flood crises that affect
millions of people, particularly in ASAL (Arid and Semi-Arid) regions.
Early warning systems can give communities and authorities days or weeks
of advance notice to prepare. This project builds an automated data
pipeline that ingests daily weather data for all 47 Kenyan counties,
detects anomalies against 40+ year historical baselines, and surfaces
risk scores on an interactive dashboard.
## Project Architecture
## Tech Stack
### Cloud Infrastructure
### Data Engineering
### Infrastructure as Code
### Containerization
### Programming Languages
### Key Libraries
- `dlt` - Data extraction and loading
- `pandas` - Data manipulation
- `pyarrow` - Efficient data processing
- `google-cloud-bigquery` - BigQuery client
- `google-cloud-storage` - Cloud Storage client
- `pydantic` - Data validation
- `tenacity` - Retry logic
- `sqlparse` - SQL parsing
## Project Structure
```text
kenya-climate-risk-monitor/
│
├── .gitignore
├── LICENSE
├── README.md
├── docker-compose.yaml
├── dockerfile
│
├── data/
│ ├── historical/
│ │ └── kenya_weather_2019.csv
│ ├── KenyaRegions.json
│ └── kenya_counties.csv
│
├── docs/
│ ├── dbt_lineage.jpg
│ ├── entity_relationship_diagram.svg
│ ├── kenya_weather_gcp_architecture.svg
│ ├── kestra_dashboard.jpg
│ └── methodology.md
│
├── extraction/
│ ├── .dlt/
│ │ └── config.toml
│ ├── .gitignore
│ ├── backfill_historical.py
│ ├── pipeline_backfill.py
│ ├── pipeline_daily.py
│ ├── pipeline_dims.py
│ ├── pipeline_weather_source.py
│ ├── retry_failed.py
│ ├── wasp_mining.py
│ └── wasp_test.py
│
├── dbt/
│ ├── .gitignore
│ ├── dbt_env/ # Python virtual environment (ignored)
│ ├── kenya_climate_risk_monitor/
│ │ ├── .gitignore
│ │ ├── README.md
│ │ ├── dbt_project.yml
│ │ ├── analyses/
│ │ …