Home Business & Research Bi-Manual Robot Teleoperation Using a RGB-D Range Sensor

Bi-Manual Robot Teleoperation Using a RGB-D Range Sensor

😎 New DIY robot kits

Here is a bi-manual teleoperation system that is controlled using input data from a cheap RGB-D range sensor (like the ASUS Xtion PRO). Such an approach would allow operators to interact with robots from a distance without having to use fancy exoskeletons and sensors. As the developers explain:

we implemented a 3D version of the OpenPose package. The first stage of our method contains the execution of the OpenPose Convolutional Neural Network (CNN), using a sequence of RGB images as input. The extracted human skeleton pose localization in two-dimensions (2D) is followed by the mapping of the extracted joint location estimations into their 3D pose in the camera frame. The output of this process is then used as input to drive the end-pose of the robotic hands relative to the human hand movements, through a whole-body inverse kinematics process in the Cartesian space

Bi-Manual Articulated Robot Teleoperation using an External RGB-D Range Sensor

The approach was tested on the Centauro robot.


*Our articles may contain aff links. As an Amazon Associate we earn from qualifying purchases. Please read our disclaimer on how we fund this site.