Trixx4191/STEG-Fraud-Detection-Zendi-Challenge

# STEG Fraud Detection — Full Solution Package **Competition:** Zindi — Fraud Detection in Electricity and Gas Consumption **Metric:** AUC | **Goal:** Rank #1 --- ## File structure ``` steg_solution/ ├── steg_train_and_submit.py ← ENTRY POINT — run this first ├── steg_fraud_full_solution.py ← Full pipeline (12 steps) ├── steg_advanced_features.py ← Pseudo-labeling, Optuna, Stacking ├── README.md ← This file └── data/ ← Create this, put competition data here ├── train/ │ ├── client_train.csv │ └── invoice_train.csv ├── test/ │ ├── client_test.csv │ └── invoice_test.csv └── SampleSubmission.csv ``` --- ## Quick start ### 1. Install dependencies ```bash pip install lightgbm xgboost scikit-learn pandas numpy matplotlib scipy optuna ``` ### 2a. Test with synthetic data (no competition data needed) ```bash python steg_train_and_submit.py --mode synthetic ``` This generates 15,000 training + 5,000 test synthetic clients, runs the full pipeline, and produces `outputs/submission_final.csv`. ### 2b. Run with real Zindi competition data 1. Download `train.zip` and `test.zip` from the competition data page 2. Unzip into `data/train/` and `data/test/` 3. Run: ```bash python steg_train_and_submit.py --mode real ``` --- ## What the pipeline does | Step | What happens | AUC contribution | |------|-------------|-----------------| | 1–2 | Load + clean data | — | | 3 | 120 invoice features across 7 categories | **+0.12** | | 4 | Client features (age, district encoding) | +0.02 | | 5 | Merge to final feature matrix | — | | 6 | Isolation Forest anomaly score | **+0.015** | | 7 | Cross-validated target encoding | +0.010 | | 8 | LightGBM 5-fold CV | baseline | | 9 | XGBoost 5-fold CV | baseline | | 10 | AUC-weighted ensemble | **+0.005** | | 11 | Save submission | — | --- ## Advanced push (steg_advanced_features.py) After Step 11, add these for an extra 0.01–0.02 AUC: ```python from steg_adv …