muhinja2002-svg/kenya-environmental-pipeline

End-to-end data collection and analysis pipeline for Kenya environmental indicators | Python, BeautifulSoup, Pandas # Kenya Environmental Data Pipeline End-to-end data pipeline built during my internship at **EcoServants** (Nairobi, Kenya). The pipeline collects publicly available environmental and climate indicators for Kenya from the World Bank Open Data API, cleans and validates the data, and produces analysis-ready reports and visualisations. --- ## What It Does ``` World Bank Open Data API | v [1] scraper.py **Note:** `EN.ATM.CO2E.PC` (CO2 emissions) may return no data for recent years due to World Bank reporting gaps for Kenya. All other indicators collect 20+ years of data (2000–2024). --- ## Sample Outputs The pipeline generates: - **6 trend charts** — one per indicator with time-series plot and shaded fill - **1 correlation heatmap** — showing relationships across all indicators - **2 CSV reports** — cleaned dataset and per-indicator summary statistics --- ## Setup & Usage ```bash # 1. Clone the repo git clone github.com cd kenya-environmental-pipeline # 2. Create a virtual environment python -m venv venv source venv/bin/activate # Windows: venv\Scripts\activate # 3. Install dependencies pip install -r requirements.txt # 4. Run the full pipeline python main.py ``` Outputs are saved to `outputs/reports/` and `outputs/charts/`. > **Windows users:** If you see Unicode errors in the console, run: > ```powershell > $env:PYTHONIOENCODING = "utf-8" > python main.py > ``` --- ## Project Structure ``` kenya-environmental-pipeline/ |-- src/ | |-- __init__.py | |-- scraper.py # Data collection (World Bank REST API) | └-- analysis.py # Cleaning, summary stats, visualisation |-- data/ | └-- raw/ # Raw CSVs (auto-generated on first run) |-- outputs/ | |-- reports/ # Cleaned CSV + indicator summary stats | └-- charts/ # PNG trend plots + correlation heatmap |-- logs/ # Scraper logs (auto-generated on first run) |-- main.py …