DimphoData/Maji-Ndogo-Agricultural-Data-Automation

A modular Python data pipeline for agricultural analysis. Integrates SQL relational data, performs automated ETL/data cleaning, and identifies optimal crop-growth conditions through functional programming. # Maji-Ndogo-Agricultural-Data-Automation A modular Python data pipeline for agricultural analysis. Integrates SQL relational data, performs automated ETL/data cleaning, and identifies optimal crop-growth conditions through functional programming. - Project Scope: Maji Ndogo Data-Automation Overview: The Maji Ndogo Data-Automation project is a data engineering and analytical study designed to optimize crop production in a developing region. The scope of this project covers the entire data lifecycle—from extracting raw, siloed data out of a relational database to providing high-level strategic recommendations for agricultural deployment. The project simulates the role of a Data Scientist/Engineer tasked with determining where the region's limited resources (automated farming technology) should be deployed to maximize yield and sustainability. - Project Objectives: Data Integration (Extraction)The initial scope involves breaking down data silos. Agricultural information was spread across four distinct database tables. This project implements a unified data structure by joining A, Geographic Data B, Field locations C, Physical Terrain Features - Weather Patterns: Historical rainfall and temperature data Soil Characteristics: Chemical composition and fertility scores Management Records: Crop types and historical yield performance - Data Quality & ETL (Transformation)A major component of this project is identifying and programmatically fixing "dirty data." The scope includes; 1, Schema Correction: Resolving a critical error where column headers (Crop_type and Annual_yield) were inverted. 2, String Standardization: Cleaning categorical data to remove "fuzzy" duplicates and spelling errors (e.g., standardizing 'teaa' to 'tea'). 3, Range Validation: Correcting geographic outliers, such as negative elevation values, to ensure geospatial accuracy. - Environmental Analysis (Insight)The project identifies the "environmental signature" of Maji Ndogo's crops. By analy …