Kevinobote/Real-Time-Swahili-Speech-Processing-System

# **Real-Time Swahili Speech Processing System** ## 📌 Abstract This project focuses on developing a **real-time Swahili speech analysis system** that integrates **speech-to-text conversion, sentiment analysis, and text summarization**. Many **low-resource languages**, including Swahili, lack robust **automatic speech recognition (ASR) models** due to limited training data. Using the **Mozilla Common Voice Swahili dataset (2022)**, we implemented a **machine learning pipeline** with pre-trained models: - **Whisper** (speech-to-text) - **DistilBERT** (sentiment analysis) - **T5** (text summarization) ### **Key Findings** ✅ **Speech Recognition:** Achieved a **Word Error Rate (WER) of 3.99**, demonstrating high transcription accuracy. ✅ **Sentiment Analysis:** Reached **60% accuracy**, showing moderate success in Swahili emotion classification. ✅ **Text Summarization:** Faced challenges, with low **ROUGE scores** (ROUGE-1: 0.0917, ROUGE-2: 0.0600, ROUGE-L: 0.0917), indicating difficulties in summarizing Swahili text effectively. To improve performance, **expanding Swahili training datasets** and **fine-tuning language-specific models** are essential. This research contributes to **bridging language barriers** and supports practical applications in **education, healthcare, and accessibility**. --- ## **🔹 Features** ✅ **Real-Time Speech-to-Text** – Converts Swahili audio into text using **OpenAI’s Whisper model**. ✅ **Sentiment Analysis** – Detects emotions in spoken language with **DistilBERT**. ✅ **Text Summarization** – Generates concise summaries using **T5 Transformer models**. ✅ **Interactive Dashboard** – Uses **Streamlit** for **real-time visualization and analysis**. ✅ **Audio Preprocessing** – Includes **noise reduction, silence trimming, and feature extraction**. ✅ **Performance Metrics** – Evaluates models using **WER, accuracy, F1-score, and ROUGE scores**. ✅ **User Feedback Loop** – Allows users to rate transcriptions and summaries to improve …