Qinisani811/audio-fraud-detector

AI-powered fraud detection system for South African insurance call centres. Analyzes voice (MFCCs, pauses, pitch) + text transcripts to flag fraud in real-time. XGBoost achieves high accuracy on test set with SHAP explainability. Built with Python, Librosa, Whisper, FastAPI, Streamlit on Jupyternotebook. AI Fraud Detection for SA ## Overview fraud detection is an AI-powered fraud detection system for South African insurance call centres. It analyzes "voice patterns "(pauses, pitch, MFCCs) and **speech transcripts** (hesitation words like "not sure", "maybe") to flag fraudulent claims in real-time. ## Problem - South Africa loses more than R2 billion annually to insurance fraud by criminalls Most of the time the fraud cases are really fraud - Traditional detection is reactive (after money is paid) ## Solution - **Real-time analysis** during customer calls - **XGBoost model** achieves very high accuracy on test set - **SHAP explainability** shows WHY a call was flagged - **Audio features**: MFCCs, chroma, spectral features, pause detection - **Text features**: hesitation word count ("not sure", "maybe", "i think") ## Tech Stack - Python, Jupyter Notebook - Librosa (audio processing) - Whisper (speech-to-text) - XGBoost (classification) - SHAP (explainability) ## Results | Model | F1 Score | -----model 1------ |-------|----------| | Logistic Regression | 0.88 | ----Model 2-- | XGBoost |0,86 | Top features driving fraud detection: - Pause length - hasitation frequency - MFCC voice characteristics - Word count ## Data - i initially started with 30 synthetic audio files (15 fraud, 15 legitimate) that i recorded my self - i saw that the model was overfitting(memorising) i than added 70 more audios taking the total to 100 audios now (50 fraud and 50 legit) - 77 audio features + 6 text features - Transcripts generated by Whisper then changed to faster-whisper due to time dependency ## Author Qinisani Ngcobo Final Year Data Science Student(MR Q)