DURableVS: Data-efficient Unsupervised Recalibrating Visual Servoing via online learning in a structured generative model
Paper in proceeding, 2022

Visual servoing enables robotic systems to perform accurate closed-loop control, which is required in many applications. However, existing methods either require precise calibration of the robot kinematic model and cameras or use neural architectures that require large amounts of data to train. In this work, we present a method for unsupervised learning of visual servoing that does not require any prior calibration and is extremely data-efficient. Our key insight is that visual servoing does not depend on identifying the veridical kinematic and camera parameters, but instead only on an accurate generative model of image feature observations from the joint positions of the robot. We demonstrate that with our model architecture and learning algorithm, we can consistently learn accurate models from less than 50 training samples (which amounts to less than 1 min of unsupervised data collection), and that such data-efficient learning is not possible with standard neural architectures. Further, we show that by using the generative model in the loop and learning online, we can enable a robotic system to recover from calibration errors and to detect and quickly adapt to possibly unexpected changes in the robot-camera system (e.g. bumped camera, new objects).

Generative modeling

visual servoing

data-afficient learning

Robotics

Manipulation

Computer science

Author

Nishad Gothoskar

Massachusetts Institute of Technology (MIT)

Miguelle Lázaro-Gredilla

Vicarious AI

Yasemin Bekiroglu

Vicarious AI

Abhishek Agarwal

Vicarious AI

Josh Tenenbaum

Massachusetts Institute of Technology (MIT)

Vikash Mansinghka

Massachusetts Institute of Technology (MIT)

George Dileep

Vicarious AI

Proceedings - IEEE International Conference on Robotics and Automation

10504729 (ISSN)

6674-6680

IEEE International Conference on Robotics and Automation
Philadelphia , USA,

Subject Categories

Robotics

Control Engineering

Computer Science

More information

Latest update

1/11/2023