Full-stack Azure Databricks data engineering + ML case study — 19 African IPP plants, 3M+ SCADA rows, 8 ML models, ADF pipeline
# Aquila Energy Intelligence Platform
## Aquila African Energy Intelligence — Qlik Sense portfolio app
The portfolio now includes a six-page Qlik Sense Desktop SCADA command experience covering fleet capacity, reliability, telemetry, commercial/ESG performance and ML trust. Download the QVF and industrial control-room extension from `qlik/portfolio`.
**Full-stack Azure Databricks data engineering + ML completed project**
Synthetic SCADA telemetry from 19 African IPP power plants · 3M+ rows · 9 ML models
---
## Project Overview
Aquila operates utility-scale power plants across seven African nations. This platform
ingests 15-minute SCADA telemetry, applies a Medallion Architecture (Bronze → Silver → Gold)
on Azure Databricks, and surfaces predictive ML models for operations, maintenance, and
commercial decision-making.
| Metric | Value |
|---|---|
| Total rows generated | **3,024,807** |
| SCADA telemetry rows | **2,981,664** (15-min intervals, 17 plants × 5 years) |
| Plants | **19** across **7 countries** (17 operating · 2 under construction) |
| Operating capacity | **1,794 MW** · Construction pipeline: **485 MW** |
| Date range | **2020–2024** (5 years) |
| ML models | **9** registered in MLflow (6 operational + 3 forecasting) |
| Annual portfolio revenue | **~R139M** (synthetic; reflects contracted PPA tariff scale) |
| Portfolio availability (operating fleet) | **~92%** (82.7% incl. under-construction plants) |
---
## Stack
- **Azure Databricks** — PySpark notebooks, Delta Lake, MLflow
- **Azure Data Factory** — Copy Activities, ForEach, IfCondition, Teams webhook
- **Azure Data Lake Gen2** — ADLS raw/bronze/silver/gold containers
- **Azure SQL Server** — AquilaEnergyDW (27 tables: 17 core + 10 V2 upgrade · 6 BI views)
- **Python** — Data generation, ML (XGBoost, LightGBM, sklearn), dirty-data injection
- **Power BI** — 111 DAX measures across 9 report pages, Databricks SQL endpoint
- **openpyxl** — 8-sheet Excel report with charts
---
## Directory …