Night-to-day image translation for retrieval-based localization
Paper in proceedings, 2019

Visual localization is a key step in many robotics pipelines, allowing the robot to (approximately) determine its position and orientation in the world. An efficient and scalable approach to visual localization is to use image retrieval techniques. These approaches identify the image most similar to a query photo in a database of geo-tagged images and approximate the query's pose via the pose of the retrieved database image. However, image retrieval across drastically different illumination conditions, e.g. day and night, is still a problem with unsatisfactory results, even in this age of powerful neural models. This is due to a lack of a suitably diverse dataset with true correspondences to perform end-to-end learning. A recent class of neural models allows for realistic translation of images among visual domains with relatively little training data and, most importantly, without ground-truth pairings.In this paper, we explore the task of accurately localizing images captured from two traversals of the same area in both day and night. We propose ToDayGAN - a modified image-translation model to alter nighttime driving images to a more useful daytime representation. We then compare the daytime and translated night images to obtain a pose estimate for the night image using the known 6-DOF position of the closest day image. Our approach improves localization performance by over 250% compared the current state-of-the-art, in the context of standard metrics in multiple categories.

Author

Asha Anoosheh

Swiss Federal Institute of Technology in Zürich (ETH)

Torsten Sattler

Chalmers, Electrical Engineering, Signalbehandling och medicinsk teknik, Imaging and Image Analysis

Radu Timofte

Swiss Federal Institute of Technology in Zürich (ETH)

Marc Pollefeys

Swiss Federal Institute of Technology in Zürich (ETH)

Luc Van Gool

Swiss Federal Institute of Technology in Zürich (ETH)

Proceedings - IEEE International Conference on Robotics and Automation

10504729 (ISSN)

Vol. 2019-May 5958-5964 8794387

2019 International Conference on Robotics and Automation, ICRA 2019
Montreal, Canada,

Subject Categories

Media Engineering

Computer Vision and Robotics (Autonomous Systems)

Medical Image Processing

DOI

10.1109/ICRA.2019.8794387

More information

Latest update

1/20/2020