anthonyapollis/DebtBusters_Intelligence_Platform

Enterprise Data Warehouse & ML Platform for South African debt counselling (Confluent/DebtBusters). Databricks Azure · Delta Lake · PySpark · MLflow · 5 ML Models · ~3.96M synthetic rows. # DebtBusters Intelligence Platform ## Confluent Financial Wellness Group — Enterprise Data Warehouse on Databricks Azure --- ## Project Overview End-to-end data engineering and ML completed project modelled on the Confluent/IDM group (DebtBusters, JustMoney). Covers the full South African debt counselling and financial wellness lifecycle: lead acquisition → affordability assessment → debt review → payment distribution → credit monitoring → debt clearance. **Tech Stack:** Databricks (Azure) · Delta Lake · PySpark · MLflow · XGBoost · LightGBM · CatBoost · Power BI --- ## Architecture ``` Google Analytics / CRM / Call Centre / Credit Bureau / PDA Payments │ ┌────▼────┐ │ Bronze │ Raw Delta ingestion + audit columns └────┬────┘ │ ┌────▼────┐ │ Silver │ Cleaned · Deduplicated · Validated · Typed └────┬────┘ │ ┌────▼────┐ │ Gold │ Star schema dims + facts + KPI views └────┬────┘ │ ┌───────────────┼───────────────┐ ┌────▼────┐ ┌─────▼─────┐ ┌────▼────┐ │ ML │ │ Power BI │ │ Databricks│ │ Models │ │ Dashboards │ │ SQL │ └─────────┘ └───────────┘ └─────────┘ ``` --- ## Synthetic Data (80,000 clients · ~3.96M rows total) | Entity | Rows | |---------------------|-------------| | Clients | 80,000 | | Leads | 500,000 | | Assessments | 120,000 | | Debt Review Cases | 60,000 | | Debt Accounts | 700,000 | | Repayment Plans | 400,000 | | Payments | 1,500,000 | | Credit Monitoring | 600,000 | | **Total** | **~3,960,000** | --- ## Star Schema **Dimensions:** Dim_Date · Dim_Client · Dim_Creditor · Dim_Counsellor · Dim_Financial_Product **Facts:** Fact_Lead · Fact_Assessment · Fact_Debt_Review_Case · Fact_Repayment_Plan · Fact_Payment · Fact_Credit_Monitoring --- ## ML Models — Validated Results | Model | Algorithm | Metric | Score | Business Interpretation | |-------|-----------|--------|-------|--------- …