Are Large-Scale 3D Models Really Necessary for Accurate Visual Localization
Journal article, 2021

Accurate visual localization is a key technology for autonomous navigation. 3D structure-based methods employ 3D models of the scene to estimate the full 6 degree-of-freedom (DOF) pose of a camera very accurately. However, constructing (and extending) large-scale 3D models is still a significant challenge. In contrast, 2D image retrieval-based methods only require a database of geo-tagged images, which is trivial to construct and to maintain. They are often considered inaccurate since they only approximate the positions of the cameras. Yet, the exact camera pose can theoretically be recovered when enough relevant database images are retrieved. In this paper, we demonstrate experimentally that large-scale 3D models are not strictly necessary for accurate visual localization. We create reference poses for a large and challenging urban dataset. Using these poses, we show that combining image-based methods with local reconstructions results in a higher pose accuracy compared to state-of-the-art structure-based methods, albeight at higher run-time costs. We show that some of these run-time costs can be alleviated by exploiting known database image poses. Our results suggest that we might want to reconsider the need for large-scale 3D models in favor of more local models, but also that further research is necessary to accelerate the local reconstruction process.

image retrieval

place recognition

Visual localization

pose estimation

image-based localization


Akihiko Torii

Tokyo Institute of Technology

Hajime Taira

Tokyo Institute of Technology

Josef Sivic

Ecole Normale Superieure (ENS)

Marc Pollefeys

Swiss Federal Institute of Technology in Zürich (ETH)

Masatoshi Okutomi

Tokyo Institute of Technology

Tomas Pajdla

Czech Technical University in Prague

Torsten Sattler

Chalmers, Electrical Engineering, Signal Processing and Biomedical Engineering, Imaging and Image Analysis

IEEE Transactions on Pattern Analysis and Machine Intelligence

0162-8828 (ISSN)

Vol. 43 3 814-829 8839843

Subject Categories

Media Engineering

Computer Vision and Robotics (Autonomous Systems)

Medical Image Processing





More information

Latest update