Kusbyteen/ghana-food-price-prediction
Domaine:
agricultureType de record:
projectdataset
Créateur:
Kus
Hôte:
An end-to-end Data Lakehouse project built on AWS and Databricks to predict food commodity prices across Ghanaian markets. This project utilizes the Medallion Architecture to transform raw market data into actionable insights for food security and economic forecasting.
````markdown
# 🇬🇭 Ghana Food Price Prediction
## 📈 A Data Lakehouse Approach to Agricultural Intelligence
An end-to-end Data Lakehouse project built on AWS and Databricks to predict food commodity prices
across Ghanaian markets. This project utilizes the **Medallion Architecture** to transform raw market
data into actionable insights for food security and economic forecasting.
---
## 📌 Overview
This project implements a modern **Data Lakehouse architecture** to ingest, process,
and analyze food commodity prices across major Ghanaian markets (e.g., Techiman, Makola, Tamale).
By leveraging **AWS S3** and **Databricks**, the system provides:
- Historical price transparency
- Machine Learning-based forecasting
- Insights to combat food price volatility
---
## 🏗️ Architecture: The Medallion Approach
The project follows the **Medallion Architecture**, ensuring data quality and lineage at every step:
```mermaid
graph LR
A[External APIs / GSS] -->|PySpark| B(Bronze: Raw)
B -->|Cleaning/UDFs| C(Silver: Standardized)
C -->|Aggregations| D(Gold: Analytics Ready)
C -->|Feature Engineering| E[ML Model: Price Forecast]
D --> F[Streamlit Dashboard]
````
### 🥉 Bronze (Raw)
* Ingests raw JSON/CSV data
* Sources: Ghana Statistical Service, WFP VAM
* Stored in AWS S3 using PySpark
### 🥈 Silver (Cleaned)
* Standardizes local units (e.g., "Olonka", "American Tin" → Kilograms)
* Handles missing values
* Removes outliers
### 🥇 Gold (Business)
* Star-schema tables optimized for BI tools
* Feature Store for Machine Learning
---
## 🛠️ Tech Stack
* **Platform:** Databricks (Free Edition)
* **Storage:** AWS S3 (Data Lake)
* **Processing:** PySpark (Spark SQL & DataFrames)
* **Data Format:** Delta Lake (ACID transactions on S3)
* **Orchestration:** Databricks Workflows / GitHub Actions
* **Modeling:** Prophet / XGBoost (Time-Series Forecasting)
* **Visualization:** Streamlit / Plotly
---
## 📍 Why This Matters
In Ghana, food inflation significantly impacts household disposa …