Controle de atuador robótico através do rastreamento ocular por câmera para assistência a pessoas com limitações físicas

Abstract

Technological advancements have paved the way for the development of increasingly sophisticated assistive methods. These technologies hold the potential to significantly improve the comfort of individuals with physical disabilities. In this context, the creation of methods that can identify user commands and control robotic actuators has emerged as an intriguing field of study. A substantial portion of research in this area employs both Commercial Off-The-Shelf (COTS) equipment and devices specifically designed for the purpose, such as electroencephalograms (EEG). This study proposes a user-friendly, non-invasive, and cost-effective method. Consequently, a methodology for identifying user commands through eye movement using a webcam and controlling a robotic actuator based on these commands is introduced. While similar works have been identified in the literature, this study seeks a low-cost development alternative. The work was conducted using the Ubuntu 20.04 operating system with Noetic ROS, and the RT-GENE packages, which were used to obtain eye tracking and blink detection, and YOLO, which aimed to detect the objects available to the user. The Python Tkinter package was employed to develop the user interface. Finally, MoveIt, Gazebo and franka_ros packages were utilized to perform the pick and place simulations of the manipulator. Simulation results using the proposed methodology are also presented, demonstrating its potential for ongoing research and real-world evaluation. Tests with the Youbot robot revealed challenges due to the need for precision and the accumulation of errors over time. However, tests with the Panda manipulator showed that the resolution of a standard webcam allowed for the determination of the object’s position in the world, despite the robot encountering difficulties in performing its task for some positions. Despite these challenges, it was possible to overcome the problem with a simple trial and error implementation. Future work aims to implement other robot models and conduct tests with real robots, in addition to adding sensing to the robot to ensure greater precision in its tasks and prevent accidents with people who are in its work area.