Amr3laa2/sign-langauge-pred

Egyptian Sign Language Prediction # Egyptian Sign Language Prediciton Using CNN And LSTM Hand gestures are one of the nonverbal communication modalities used in sign language. It is most often used by deaf people who have hearing or speech impairments to communicate with other deaf people or even with normal people, so a Deep Learning model is implemented to predict the signs and translate them accurately and that is our mission to help them. The model has been trained over a huge number of labelled videos and processed by Convolutional and LSTM layers. The dynamic visuals were extracted as points in three dimensions by MediaPipe. After using these methodologies in our model, we got a very efficient model with 100% training and validation accuracy, 99.98% in testing, and very accurate results in real-time testing. ## The main challenges facing the purpose of the application are: - The speed of sign recognition - The accuracy of detecting the sign gesture to its matched word - Connecting the model with a working web application ## Implementation Details : - First of all, the Data Set is manually collected. We gathered clips from certified references on popular platforms like YouTube - We then trimmed the parts of needed exact gestures using media editing software like Adobe Premiere. - Then a Fix_Video() function is applied to obtain a fixed frame rate on all videos, the chosen frame rate is 30 FPS. The function Removes excessive frames from videos that exceeds the specified number of FPS, and it duplicates frames in videos that are short of the required FPS. ```python def fixVideo(frames,video_name,startFrames=0,endFrames=0,middleFrames=0): ``` - To be able to identify signs, a very large database is required. Better results are guaranteed with a larger database (if the videos are correctly set). The video augmentation has been done with Python. Then Augmentation on the videos were applied in order to add to the training efficiency, a list of 9 different augmentations were added to the videos …