Sistema automático de decodificación de lengua de señas ecuatoriano a comunicación verbal.

Abstract

People with hearing and speech disabilities rely on sign language to express their ideas, emotions, and needs. This form of gestural communication employs hand movements and facial expressions to effectively convey messages. However, the lack of knowledge of this language on the part of the hearing population limits the opportunities for interaction and mutual understanding. For this reason, an automatic system capable of translating and learning the Ecuadorian sign language into verbal communication using artificial intelligence was developed. The project was developed based on the "Gabriel Román" Ecuadorian Sign Language Dictionary " to establish the basic vocabulary of the system, which consists of the Ecuadorian dactylological alphabet, numbers and basic words. An electronic glove for the right hand was built, with five Flex sensors, an MPU6050 inertial sensor and an Arduino Nano board, which were responsible for recording the movements of the hand and fingers, such as flexion, extension and inclination, these Data is processed using Python to create the database, in order to apply machine learning algorithms. The algorithms studied are k-Nearest Neighbors (KNN) and Random Forest, with training accuracies of 94,62% and 96,20% respectively. Finally, applying the Random Forest model, a graphical interface was created in Streamlit, where the user can visualize the translation of the sign language through text and an audio output.