Home Business & Research Bi-Manual Robot Teleoperation Using a RGB-D Range Sensor

Bi-Manual Robot Teleoperation Using a RGB-D Range Sensor

33
0
🤖 check out more robots like this

Here is a bi-manual teleoperation system that is controlled using input data from a cheap RGB-D range sensor (like the ASUS Xtion PRO). Such an approach would allow operators to interact with robots from a distance without having to use fancy exoskeletons and sensors. As the developers explain:

we implemented a 3D version of the OpenPose package. The first stage of our method contains the execution of the OpenPose Convolutional Neural Network (CNN), using a sequence of RGB images as input. The extracted human skeleton pose localization in two-dimensions (2D) is followed by the mapping of the extracted joint location estimations into their 3D pose in the camera frame. The output of this process is then used as input to drive the end-pose of the robotic hands relative to the human hand movements, through a whole-body inverse kinematics process in the Cartesian space

The approach was tested on the Centauro robot.

[HT]

*some of our articles have aff links. Please read our disclaimer on how we fund Exxponent.