svarshanp/accent-detection

Accent Detection AI transcribes speech with OpenAI Whisper, extracts audio features, and classifies English accents (US, UK, Indian, Australian, Canadian, Scottish, Irish, African) using SVM, Random Forest, XGBoost, CNN, LSTM, and a custom Audio Spectrogram Transformer. A polished Streamlit app visualizes the predictions and audio analyses. Add Speech Technology & Transformer Model to Accent Detection Background The current project ("Text and speech PROJECT") is an Accent Detection system with: ML models: SVM, Random Forest, XGBoost (on hand-crafted features) DL models: 1D-CNN (Mel-spectrogram), Bi-LSTM (MFCC sequences) Streamlit app with upload-based accent prediction We will add two major features: Speech Technology — Live microphone recording + Speech-to-Text transcription (OpenAI Whisper) Transformer Model — Audio Spectrogram Transformer (AST) for accent classification User Review Required IMPORTANT Transformer Model Choice: We'll implement a custom Audio Spectrogram Transformer (AST) that splits Mel-spectrograms into patches and applies multi-head self-attention. This runs on your existing synthetic data (no external pretrained model download needed). This keeps training fast and avoids large downloads. IMPORTANT Speech-to-Text Engine: We'll use OpenAI Whisper (openai-whisper package) for speech transcription. This requires ~1GB model download (base model) on first use. Alternatively, we can use whisper-tiny (~75MB) for faster but slightly less accurate results. WARNING Live Microphone: Streamlit's st.audio_input() widget (Streamlit ≥1.33) will be used for browser-based microphone recording. This is simpler and more reliable than streamlit-webrtc. Users click Record → speak → get accent + transcription. Proposed Changes 1. Dependencies [MODIFY] requirements.txt Add new dependencies: diff +# Transformer Model +transformers>=4.35.0 + +# Speech-to-Text (Whisper) +openai-whisper>=20231117 +ffmpeg-python>=0.2.0 2. Transformer Model — Training [NEW] train_transformer.py Implements the Audio Spectrogram Transformer (AST): Architecture: Splits Mel-spectrograms into patches → Linear patch embedding → Positional encoding → Transformer Encoder (4 layers, 4 heads) → Classification head Input: Same mel_spec.npy data used by CNN (shape: N × 128 × 94) Patch size: 16×16 patches from the spectrogram Trai …