Home Drones & Surveillance Gaze-Controlled AI Drone Developed

Gaze-Controlled AI Drone Developed

🤖 check out more robots like this

Here is an AI drone that can be controlled only using your gaze. Researcher from the the University of Pennsylvania and New York University have developed a deep learning system using NVIDIA GPUs that enable pilots to control drones by moving their eyes.

Human Gaze-Driven Spatial Tasking of an Autonomous MAV voice

They used a set of glasses with a gaze tracker, camera, and IMU to estimate the position of the operators with respect to the quadcopter to allow them to navigate a drone in a new environment. The drone doesn’t need additional external sensors to work. As explained on NVIDIA’s blog:

Using NVIDIA GeForce GTX 1080 Ti GPUs and the cuDNN-accelerated TensorFlow deep learning framework, the team trained their neural networks to compute the 3D navigation waypoint from the 2D gaze coordinate provided from the glasses. They also trained the networks to perform object detection.


*Our articles may contain aff links. As an Amazon Associate we earn from qualifying purchases. Please read our disclaimer on how we fund this site.