MaryemElyazghi/COVID-19-Morocco-Analytics-Pipeline

End-to-end cloud data pipeline processing COVID-19 Morocco data — from API ingestion through PySpark transformation to Power BI dashboards with 15+ KPIs. # 🦠 COVID-19 Morocco Analytics Platform > Full-stack cloud data platform with RESTful APIs, processing COVID-19 Morocco data — from API ingestion through PySpark transformation to FastAPI + Power BI dashboards with 15 KPIs. --- ## 📋 Table of Contents - Architecture - Features - Tech Stack - Quick Start - REST API - Project Structure - Pipeline Stages - Data Quality - Dashboards & KPIs - Airflow Orchestration - CI/CD - Configuration - Testing --- ## 🏗 Architecture ``` ┌──────────────┐ ┌──────────────────┐ ┌─────────────────┐ ┌────────────┐ ┌─────────────┐ │ COVID-19 │───▶│ Azure Data │───▶│ Databricks / │───▶│ PostgreSQL │───▶│ Power BI │ │ API Source │ │ Factory (ADF) │ │ PySpark Local │ │ Database │ │ Dashboards │ └──────────────┘ └──────────────────┘ └─────────────────┘ └─────┬──────┘ └─────────────┘ │ │ │ │ Delta Lake │ │ Storage ┌─────┴──────┐ │ │ │ FastAPI │ ┌──────────────┐ ┌──────────────┐ │ REST API │──▶ React / Power BI │ Airflow │ │ Data Quality │ │ /api/v1/* │ │ Orchestrator │ │ Framework │ └────────────┘ └──────────────┘ └──────────────┘ ``` **Local Mode:** The pipeline runs entirely on your machine. PySpark simulates Databricks, SQLite replaces PostgreSQL, and FastAPI serves data to dashboards. --- ## ✨ Features | Feature | Description | |---------|-------------| | **API Ingestion** | Automated data extraction from COVID-19 APIs with retry logic & rate limiting | | **RESTful API** | FastAPI backend with 20+ endpoints serving KPIs, trends, and regional data | | **PySpark Processing** | Scalable transformations with Delta Lake for ACID transactions | | **15 KPIs Dashboard** | Mortality rates, regional trends, vaccination progress, 7-day rolling averages | | **Data Quality** | Sch …