Lehlohonolo-Saohatse/Data-Engineering-Sales-Analytics-Using-Microsoft-Fabric-Project

A production-grade data engineering portfolio project using Microsoft Fabric, built on real South African retail data. It covers Bronze, Silver, Gold medallion architecture, PySpark transformations, and Power BI dashboards. # 🛒 SA Retail Analytics - Microsoft Fabric End-to-End Data Engineering > A **production-grade, end-to-end data engineering portfolio project** using Microsoft Fabric. > Built on a realistic South African retail dataset - Bronze → Silver → Gold medallion architecture, PySpark transformations, automated pipeline, and a live Power BI dashboard. --- ## 📸 Project Overview | Phase | Component | Status | |---|---|---| | 1 | Workspace + Lakehouse setup | ✅ | | 2 | Bronze ingestion via Dataflow Gen2 | ✅ | | 3 | Silver cleaning via PySpark notebook | ✅ | | 4 | Gold analytics (7 tables) | ✅ | | 5 | Automated pipeline (daily, 06:00 SAST) | ✅ | | 6 | Power BI dashboard (4 pages, DirectLake) | ✅ | --- ## 🏗️ Architecture ``` ┌─────────────────────────────────────────────────────────────────────┐ │ Sales_Pipeline (daily, 06:00 SAST) │ │ │ │ ┌─────────────────┐ ┌───────────────────┐ ┌───────────────┐ │ │ │ BRONZE LAYER │───▶│ SILVER LAYER │───▶│ GOLD LAYER │ │ │ │ Dataflow Gen2 │ │ PySpark Notebook │ │ PySpark NB │ │ │ │ │ │ │ │ │ │ │ │ customer.csv │ │ silver_customer │ │ gold_customer│ │ │ │ product.csv │ │ silver_product │ │ gold_product │ │ │ │ orders.csv │ │ silver_orders │ │ gold_segment │ │ │ └─────────────────┘ └───────────────────┘ │ gold_category│ │ │ ▲ │ gold_trends │ │ │ │ │ gold_stores │ │ │ JSON API (GitHub) │ gold_payments│ │ │ SA retail data └──────┬───────┘ │ └──────────────────────────────────────────────────────────┼─────────┘ │ DirectLake │ ┌────────────▼──────────┐ │ Power BI Dashboard │ │ 4 pages • ZAR KPIs │ └────────────────────── …