Sistema adaptativo para teleoperação de basemóvel através de reconhecimentos gestuais

Abstract

Teleoperated robots are generally used for operations in environments that are difficult to access or in places where the lives of operators are at risk. A teleoperation system uses motion interface approaches that allow remote commands to be sent to the robotic mobile base. These teleoperation methods seek to achieve aspects of stability and telepresence. However, these traditional equipment use specific components and often attached to the operator’s body, which can make it difficult to move and teleoperate, in addition to making it difficult to leave the place in case of danger. In this sense, this dissertation aims to present the development of an adaptive and intuitive system for teleoperation of a mobile base coupled with a robotic arm with three degrees of freedom. To develop this system, technologies for detecting key points of the human body are presented through deep learning techniques extracted through an RGB image. These techniques were used during the development of this work in other researches for the area of teleoperation that culminated in the technology used for this work. This work makes an approach of the entire structure of equipment, sensors, adaptations carried out in the Beckman Coulter ORCA, being a robotic manipulator of three degrees of freedom, as well as all the ROS (Robot Operating System) packages of communication developed for the application and accomplishment of the experiments. This project uses the holistic pipeline of the MediaPipe framework to capture 2D points of the operator’s body position through the images and two algorithms are developed through this framework. The first algorithm is responsible for extracting characteristics from the operator performing the requested movement to execute a given movement process.These features are used to train an SVM (Support Vector Machine) classifier, where each gesture is linked to a movement class. The second algorithm is responsible for using the data collected from the operator’s body at process time and identifying, through classification, the movement requested by the operator. After classifying the movement, the position of the key point is calculated, which, through techniques proposed by the algorithm of this work, results in a value from 0 to 100 of activation of the required movement. This value passes through a Fuzzy control system, which outputs the robot’s movement. The tests are carried out with 20 volunteer operators in order to follow a trajectory and collect/deliver an object. The evaluation of the proposed teleoperation system is carried out through experimentation in a simulated environment.Experiments were conducted to show the benefits of the proposed solutions.