sylvia772/africa-microfinance-data-pipeline

# Africa Microfinance Data Pipeline An end-to-end ELT data pipeline for a multi-country African microfinance operation spanning Nairobi (Kenya), Accra (Ghana), and Kigali (Rwanda) across 2020-2023. Built as a portfolio project to demonstrate analytics engineering and data engineering skills using modern data stack tools. --- ## Why This Project Exists In microfinance operations across Africa, loan data is often collected manually by field officers using Excel spreadsheets. This creates significant data quality challenges — inconsistent formats, missing values, schema drift across years, and mixed currencies — that make it nearly impossible to answer critical business questions reliably. **The cost of bad data in lending is not just technical — it's financial and human.** When data is unreliable: - Credit managers make loan approval decisions based on wrong signals - ML models trained on dirty data learn the wrong patterns and misclassify borrowers as low risk when they are high risk - Branch managers cannot identify which loan officers are approving risky loans - Executives cannot compare performance across cities because metrics are calculated inconsistently - Borrowers who are creditworthy get rejected while risky borrowers get approved **The goal of this pipeline is to turn messy operational data into a single source of truth that analysts, ML engineers, and business leaders can trust.** Clean, well-structured data is not just a technical achievement — it is the foundation that every downstream decision, model, and insight depends on. A dashboard is only as trustworthy as the pipeline behind it. An ML model is only as accurate as the data it was trained on. This pipeline applies the medallion architecture (Bronze → Silver → Gold) to systematically address data quality at every layer before data reaches any consumer. --- ## What This Project Does Raw Excel operational data from microfinance loan officers is transformed into clean, business-ready table …