ELIASTAFADZWAMUDZUDZDU/Zimbabwe-Education-Master-Data-Platform

An end-to-end Data Engineering platform for Zimbabwe secondary school data featuring automated ETL, PostgreSQL, Streamlit dashboard, and an integrated AI assistant for natural language database exploration. # 🏫 Zimbabwe Education Master Data Platform (ZEMDP) *A centralized education data platform built using modern Data Engineering practices, enhanced with an integrated AI assistant for natural language database exploration.* --- ## 📖 Overview The **Zimbabwe Education Master Data Platform (ZEMDP)** is an end-to-end Data Engineering project designed to collect, transform, integrate, store, and visualize Zimbabwe secondary school information from multiple public data sources. The platform automates the complete ETL (Extract, Transform, Load) process, consolidating fragmented educational data into a centralized PostgreSQL database. Users can explore the data through an interactive Streamlit dashboard featuring real-time statistics, search, filtering, and an integrated AI assistant capable of answering natural language questions about the database. --- # 🎯 Project Objectives The project was developed to demonstrate modern Data Engineering principles by: - Designing a complete ETL pipeline - Integrating data from multiple public sources - Cleaning and standardizing inconsistent datasets - Building a normalized PostgreSQL database - Developing an interactive analytics dashboard - Integrating an AI assistant for natural language database exploration - Applying software engineering best practices for modular and maintainable code --- # ✨ Features ## 🔹 Data Engineering - Automated ETL Pipeline - Multi-source Data Integration - Data Cleaning - Data Standardization - Duplicate Detection - Missing Value Handling - Data Validation - Master Dataset Generation --- ## 🔹 Database - PostgreSQL Relational Database - Normalized Database Design - Database Views - ETL Execution Logging - Data Quality Monitoring --- ## 🔹 Dashboard Interactive Streamlit dashboard providing: - Dashboard Statistics - School Directory - Search Functionality - Province Filter - Education Level Filter - Responsive Data Tables --- ## 🔹 AI Assistant The platform includes an integrated AI assis …