NovaDataAnalytics/retail-data-quality-casestudy

How dirty data cost a South African retailer R2.9M in phantom revenue — a data cleaning case study # Retail Data Quality Case Study ### How Dirty Data Cost a Retailer R2,985,788 in Phantom Revenue --- ## Overview This project demonstrates how common data quality issues in retail transaction data lead to catastrophically incorrect revenue reporting — and how systematic data cleaning resolves them. Using a realistic South African retail dataset of 1,050 transactions across 10 product SKUs and 5 regions, we show that uncleaned data reported **R7,129,011** in revenue for FY 2024. The true figure after cleaning was **R4,143,223** — a **R2,985,788 overstatement (72.1% error)**. This is a demonstration project built to illustrate real data quality problems encountered across South African SMEs, NGOs, and public sector organisations. --- ## The Problem Most businesses export transaction data and aggregate it directly — without validating or cleaning it first. This analysis shows exactly what that costs. | Metric | Dirty Data | Clean Data | |---|---|---| | Total Rows | 1,050 | 1,000 | | Reported Revenue | R7,129,011 | R4,143,223 | | Overstatement | R2,985,788 | — | | Error Rate | 72.1% | 0% | --- ## Data Quality Issues Covered | Issue | Rows Affected | Business Impact | |---|---|---| | Duplicate transactions | 50 rows | Direct revenue double-counting | | Missing values | 80 cells | Breaks segmentation and reporting | | Inconsistent region formatting | 79 rows / 22 variants | Breaks all regional aggregation | | Inconsistent date formats | 59 rows / 5 formats | Breaks time-series analysis | | Prices stored as strings with R symbol | 11 rows | Silent calculation failures | | Discount rates entered as % not decimal | 9 rows | Inflates/deflates revenue | | Negative revenues (unreconciled returns) | 15 rows | Distorts revenue totals | | Inflated amounts (data entry errors) | 7 rows | Massive revenue overstatement | | Zero revenues (missing calculations) | 10 rows | Understates true revenue | --- ## Repository Structure ``` retail-data-quality-casestudy/ │ ├─ …