theoedibo/Climate-Impact-on-Crop-Availability-Automated-ETL-Pipeline

Domain:

agricultureclimate

Record type:

software

Creator:

the

Host:

This project builds a fully automated ETL (Extract, Transform, Load) pipeline to analyze how weather conditions affect crop availability across selected regions in Nigeria. # 🌾 Climate Impact on Crop Availability – Automated ETL Pipeline ## 📌 Overview This project builds a fully automated **ETL (Extract, Transform, Load) pipeline** to analyze how weather conditions affect crop availability across selected regions in Nigeria. The system extracts historical weather data and combines it with crop yield data to uncover patterns between **climate variables (temperature & rainfall)** and **agricultural productivity**. --- ## 🎯 Objective To determine how weather conditions influence crop yield and availability using a reproducible, automated data pipeline. --- ## 🧠 Key Features * ✅ Modular ETL pipeline (Extract → Transform → Load) * ✅ Automated weekly execution (Windows Task Scheduler) * ✅ Incremental data loading (append, not overwrite) * ✅ Data cleaning & validation * ✅ SQLite database storage * ✅ Logging system for monitoring * ✅ Exploratory data analysis & visualization * ✅ Machine learning prediction model * ✅ Streamlit dashboard for interactive insights --- ## 📥 Data Sources ### 🌦️ Weather Data * Source: Open-Meteo Historical API * Endpoint: archive-api.open-meteo.com **Features:** * time * temperature * precipitation * region * year --- ### 🌾 Crop Data * Source: Synthetic dataset (generated programmatically) **Features:** * date * region * crop (Maize, Rice, Sorghum) * yield * year --- ## 🧱 Data Schema ### Crop Dataset | Column | Description | | ------ | ---------------- | | date | Observation date | | region | Nigerian state | | crop | Crop type | | yield | Daily yield | | year | Year | ### Weather Dataset | Column | Description | | ------------- | ------------------- | | time | Observation date | | temperature | Average temperature | | precipitation | Rainfall | | region | Nigerian state | | year | Year | --- ## 🔄 Pipeline Architecture Crop Data + Weather Data → Extract → Cl …

Visit

github.com

KutliMolefe/Climate-Health-ETL-Pipeline-Dashboard-

Developed an end-to-end ETL pipeline to analyze correlations between climate data and disease outbre

Benness001/weather-etl-pipeline

Automated hourly ETL pipeline for Lagos, Nigeria weather data — Python, SQL, Apache Airflow, Snowfla

Thiyane24/etl-pipeline-moz-economy

End-to-end ETL pipeline using medallion architecture (Bronze/Silver/Gold) on Mozambique World Bank e

c-matias/petroleum-etl-pipeline

End-to-end data pipeline ingesting, validating and transforming petroleum production data from Angol

lawrykoomson/ghana-weather-etl-pipeline

End-to-end weather data pipeline for 5 Ghanaian cities — Airflow orchestrates extraction from Open-M

manasseh-asaah/etl-data-pipeline

Modular ETL pipeline: extracts African country data from a REST API, transforms and validates it, lo