abdullahsherdy/ESL-Egyptian-Sign-Language-Recognition

This project presents a real-time Egyptian Sign Language (ESL) recognition system that translates sign language gestures into text using a webcam. The system combines hand gesture recognition and facial emotion detection to better capture the linguistic and expressive components of ESL. # Egyptian Sign Language (ESL) Real-Time Recognition System ## Project Overview This project presents a **real-time Egyptian Sign Language (ESL) recognition system** that translates sign language gestures into text using a webcam. The system combines **hand gesture recognition** and **facial emotion detection** to better capture the linguistic and expressive components of ESL. The project is designed as a **graduation-level prototype**, focusing on feasibility, clarity, and real-time performance rather than large-scale deployment. --- ## Objectives * Recognize **word-level Egyptian Sign Language signs** from live webcam input * Integrate **facial emotion detection** as a non-manual linguistic cue * Run **in real time** on a local machine * Use **pre-trained models and lightweight classifiers** (no training from scratch) * Deliver a **working demo application** suitable for academic evaluation --- ## System Architecture (High Level) ``` Webcam Input ↓ Frame Capture (OpenCV / Streamlit) ↓ MediaPipe Holistic (Hand + Face Landmarks) ↓ Feature Extraction ↓ Sign Classifier (MLP / LSTM) ↓ Predicted Word ↓ Emotion Detection (DeepFace) ↓ Live Display (Text Overlay) ``` --- ## Dataset * **Dataset:** Egyptian Sign Language (ESL) * **Structure:** * Word-level classes * Multiple videos per word * **Usage:** * Videos are split into frames * Hand landmarks are extracted using MediaPipe * Facial emotion is inferred using a pre-trained model Dataset link: data.mendeley.com > ⚠️ Note: The dataset contains **sign labels only** (no facial expression labels). > Facial emotion is inferred using pre-trained emotion models. --- ## Tech Stack ### Core Technologies * **Python 3.9+** * **OpenCV** – webcam capture & visualization * **MediaPipe** – hand & face landmark detection * **TensorFlow / Keras** – sign classification model * **DeepFace** – facial emotion recognition * **NumPy / Pandas** – data processing ### Demo Interface (not decided …