zanaicompanyltd/nena

Nena is an AI-powered bidirectional mobile communication platform that eliminates communication barriers for Deaf and hard-of-hearing individuals across East Africa. It enables real-time, two-way translation between Sign Language (TSL/KSL) and spoken Swahili both face-to-face and during live phone calls without needing a human interpreter # Nena — Sign Language to Swahili Voice App **TSL / KSL → Real-Time Swahili Speech Synthesis** > *"Nena" means "Speak" in Swahili.* ## Project Description Nena is an AI-powered bidirectional mobile communication platform that eliminates communication barriers for Deaf and hard-of-hearing individuals across East Africa. It enables real-time, two-way translation between Tanzanian/Kenyan Sign Language (TSL/KSL) and spoken Swahili — both face-to-face and during live phone calls — without needing a human interpreter. The app turns any Android smartphone into a pocket interpreter by combining: - **Few-shot CLIP** gesture recognition (on-device, privacy-preserving) - **OpenAI Whisper** speech recognition - **Neural text-to-speech** with a user-selectable Male/Female voice profile - **WebRTC/SIP-based live calling** (Nena Call) for real phone conversations ### What Nena Does 1. Opens the front camera and streams video frames. 2. Sends frames to the AI backend for gesture classification (few-shot CLIP model). 3. Receives a predicted Swahili word (e.g. *"Habari"*). 4. Speaks it aloud using the user's chosen Male or Female voice profile. 5. For incoming speech (Nena Call or in person), transcribes Swahili audio via Whisper and displays it as sign animation for the Deaf user. ### Why It Matters - No affordable interpreter access excludes Deaf individuals from hospitals, banks, and government services. - Standard phone networks carry only audio — Deaf users can't call independently. - TSL/KSL have very little annotated training data, requiring a few-shot learning approach rather than standard deep learning. - Nena is designed to run on constrained 3G networks and mid-range Android hardware (2019+), with all gesture-recognition inference happening **on-device** — no video is ever uploaded, preserving user privacy. Target MVP release: **Android, Q4 2026** (iOS planned for a later phase). --- ## Directory Structure ``` nena/ ├── app/ # Flutter mo …