RighteousW/sign_avatar

Real-time bidirectional translation between speech and Namibian Sign Language using computer vision and ML. # Sign Language Translation System A real-time bidirectional translation system between speech and **Namibian Sign Language (NSL)**, combining speech recognition, computer vision, and avatar-based visualization. ## Key Features * 🎤 **Speech → Sign Translation**: Convert spoken English into sign language visualization * 📷 **Sign → Text Recognition**: Real-time sign detection using computer vision * 🔁 **Bidirectional Pipeline**: Full loop from speech ↔ text ↔ sign * 🖥️ **Unified Interface**: Single desktop app (PyQt6) integrating all components * ⚙️ **Modular Design**: Six independent but connected processing stages --- ## Architecture The system is composed of six modular components: | Component | Description | Status | | --------------------- | ------------------------------------ | ------ | | Speech → Text | Real-time speech transcription | ✅ | | Text → Gloss | English to NSL gloss translation | ✅ | | Gloss → Visualization | Avatar-based sign rendering | ✅ | | Video → Gloss | Sign recognition via computer vision | ⚠️ | | Gloss → Text | Gloss to English translation | ⚠️ | | Text → Speech | Speech synthesis | ✅ | ### Translation Flow * **Forward**: Speech → Text → Gloss → Visualization * **Reverse**: Video → Gloss → Text → Speech --- ## Installation ```bash git clone github.com cd sign_avatar pip install -r requirements.txt ``` --- ## Usage ```bash pip install -e . unified-demo ``` This launches a desktop application with: * Speech-to-sign translation * Sign-to-text recognition * Real-time visualization interface > ⚠️ Note: Speech-to-text and text-to-speech require an internet connection. --- ## Technical Highlights * Designed and implemented a **multi-stage ML pipeline** integrating speech, vision, and language components * Built a **modular architecture** t …