Gesture-Based Touchless Operations: Leveraging MediaPipe and OpenCV

Özet

Humans have only recently begun using hand gestures to interact with computers. The integration of the real and digital worlds is the aim of gesture recognition. It is considerably simpler to convey our intents and ideas to the computer via hand gestures. A simple and efficient touchless method of interacting with computer systems is through hand gestures. However, the limited end-user adoption of hand gesture-based systems is mostly caused by the significant technical challenges involved in successfully identifying in-air movements. Image recognition is one of the many ways that a computer may identify a hand gesture. The recognition of human movements is enabled through the implementation of a convolutional neural network (CNN). Within this study, we develop a simple hand tracking method for controlling a surveillance car operating on the Robot Operating System (ROS) by utilizing socket programming. Our model was trained on an extensive dataset consisting of over 3000 photographs, encompassing a wide range of letter configurations from A to Z and numbers 1 to 9. The developed algorithm demonstrates promising implications for individuals with disabilities, including those who are deaf or have speech impairments. Moreover, its versatility extends to public environments such as airports, train stations, and similar locations, offering potential for practical implementation. This approach leverages Google MediaPipe, a machine learning (ML) pipeline that incorporates Palm Detection and Hand Landmark Models. In the investigation, steering speed and direction of a ROS automobile are controlled. Vehicles for surveillance that can be operated using hand gestures may help to enhance security measures.