Sistema domótico para personas con capacidad limitada de movimiento en la extremidad superior derecha utilizando reconocimiento de gestos de la mano y algoritmos de Machine Learning

Abstract

People with physical disabilities (PD) have problems carrying out daily activities, affecting their independence. In this context, a person with PD can communicate through hand gestures or facial gestures, among others. However, selecting the features and patterns that separate one gesture from another is not a trivial problem. In this sense, a real-time domotic system (DS) that works with three subsystems is proposed. The first subsystem recognizes hand gestures using a machine learning model and infrared information. The machine learning model consists of preprocessing, feature extraction, classification, and postprocessing modules. The second subsystem relays the message between the subsystems. Finally, the third subsystem activates the operation of the actuators by gestures. The HGR model was trained using 6720 observations and tested offline with 1680 observations, giving an accuracy rate of 92.759%. additionally, the DS was tested online with 1500 observations from 10 users whose persons were not part of the input dataset or model testing. Online testing gives an accuracy rate of 84.07%. Once the hand gesture is recognized, it is sent wireless to an Esp8266 board through the MQTT protocol. The Esp8266 board activates the operation of several actuators. The mosquitto broker embedded in a raspberry pi manages the sending and receiving of messages between the computer where the gesture is recognized and the Esp8266 board. The theoretical response time of the DS is 118.87 ms.