Leomezjayson/Audio-Visual-Fusion-African-Roads

Dynamic Audio-Visual Fusion for Hazard Prediction in Unstructured Environments # Audio-Visual-Fusion-African-Roads Dynamic Audio-Visual Fusion for Hazard Prediction in Unstructured Environments # SenseAI Research — Audio-Visual Fusion for African Road Safety > **"Hearing dangers before they are seen."** --- ## 📌 Overview This repository contains the research code for my undergraduate thesis: **"Dynamic Audio-Visual Fusion for Proactive Collision Avoidance in Unstructured African Traffic Environments."** **The Problem:** Existing autonomous vehicle perception systems (trained on Western datasets like nuScenes and Waymo) fail in African road conditions characterized by: - Unpredictable pedestrian and motorcycle behavior - Obstructed visibility (dust, rain, nighttime) - Chaotic lane discipline - High ambient noise **Our Hypothesis:** Fusing vision with **real-time audio cues** (emergency sirens, tire screeches, breaking glass, honk patterns) can improve hazard detection accuracy in low-visibility scenarios by over 15%. **Key Contribution:** We propose a novel, lightweight fusion mechanism that dynamically increases audio weighting when visual confidence falls below a learned threshold. --- ## 🗺️ Roadmap - [x] Repository Setup - [ ] Data Collection Pipeline (Local African Traffic Recording) - [ ] Audio-Visual Dataset Annotation - [ ] Baseline Vision Model (ResNet50 + YOLOv8) - [ ] Baseline Audio Model (VGGish + Audio Transformers) - [ ] Late Fusion Implementation - [ ] Dynamic Confidence-Based Fusion (Novel Contribution) - [ ] Edge Deployment (TensorFlow Lite / ONNX) - [ ] Thesis Submission - [ ] Conference Paper Submission (ICLR Africa / CVPR Workshop) --- ## 🏗️ Architecture Overview ``` ┌─────────────────────────────────────────────────────────────────┐ │ SENSEAI RESEARCH PIPELINE │ ├─────────────────────────────────────────────────────────────────┤ │ ┌─────────────────┐ ┌─────────────────┐ │ │ │ Vision Encoder │ │ Audio Encoder │ │ │ │ (ResN …