Dread2005/SA-Water-Dam-Level-Predictor

This is a forecasting system designed to predict South African dam water levels 30, 60 and 90 days ahead using historical dam data. # SA Water Dam Level Predictor > 30 / 60 / 90-day XGBoost forecasts for South African rain levels in mm with dam points and current dam levels, > visualised on an interactive Folium map. --- ## Quickstart ```bash # 1. Clone and enter git clone github.com /sa-dam-predictor.git cd sa-dam-predictor # 2. Virtual environment python -m venv .venv && source .venv/bin/activate # Windows: .venv\Scripts\activate # 3. Dependencies pip install -r requirements.txt # 4. Install Playwright's browser (one-time) playwright install chromium # 5. Scrape this week's dam levels (all provinces) python -m src.scrapers.dws_scraper # 6. Download CHIRPS rainfall (2000-2024 — takes a while, ~4 GB) python -m src.scrapers.climate_downloader --start 2000 --end 2024 ``` --- ## Repo structure ``` sa-dam-predictor/ ├── config.py ← all paths, URLs, constants ├── data/ │ ├── raw/ │ │ ├── dam_levels/ ← DWS weekly scrapes (CSV + master parquet) │ │ ├── rainfall/ ← CHIRPS monthly NetCDF rasters │ │ └── temperature/ ← WorldClim / ERA5 │ └── processed/ ← merged, feature-engineered DataFrames ├── notebooks/ │ └── 01_eda.ipynb ├── src/ │ ├── scrapers/ │ │ ├── dws_scraper.py ← Playwright scraper for DWS WRMS │ │ └── climate_downloader.py │ ├── features/ │ │ └── engineer.py ← lag features, rolling windows, spatial joins │ ├── models/ │ │ └── trainer.py ← TimeSeriesSplit + XGBoost pipeline │ └── viz/ │ └── map_builder.py ← Folium interactive map ├── models/ ← serialised joblib models (gitignored) ├── figures/ ← saved chart outputs ├── logs/ └── requirements.txt ``` --- ## Scraper calibration (important — do this first) The DWS WRMS page is JS-rendered via ASP.NET. Before running the scraper at scale, verify the selectors: ```bash python -m src.scrapers.dws_scraper --province "Western Cape" --debug ``` This prints the …