Towards Robust Visual Localization in Challenging Conditions
Licentiate thesis, 2019

Visual localization is a fundamental problem in computer vision, with a multitude of applications in robotics, augmented reality and structure-from-motion. The basic problem is to, based on one or more images, figure out the position and orientation of the camera which captured these images relative to some model of the environment. Current visual localization approaches typically work well when the images to be localized are captured under similar conditions compared to those captured during mapping. However, when the environment exhibits large changes in visual appearance, due to e.g. variations in weather, seasons, day-night or viewpoint, the traditional pipelines break down. The reason is that the local image features used are based on low-level pixel-intensity information, which is not invariant to these transformations: when the environment changes, this will cause a different set of keypoints to be detected, and their descriptors will be different, making the long-term visual localization problem a challenging one.

In this thesis, four papers are included, which present work towards solving the problem of long-term visual localization. Three of the articles present ideas for how semantic information may be included to aid in the localization process: one approach relies only on the semantic information for visual localization, another shows how the semantics can be used to detect outlier feature correspondences, while the third presents a sequential localization algorithm which relies on the consistency of the reprojection of a semantic model, instead of traditional features. The final article is a benchmark paper, where we present three new benchmark datasets aimed at evaluating localization algorithms in the context of long-term visual localization.

Visual localization

long-term localization

self-driving cars

camera pose estimation


autonomous vehicles

Opponent: Magnus Oskarsson, Lunds tekniska högskola, Sverige


Carl Toft

Chalmers, Electrical Engineering, Signalbehandling och medicinsk teknik, Imaging and Image Analysis

Long-term Visual Localization using Semantically Segmented Images


Paper in proceedings

Semantic Match Consistency for Long-Term Visual Localization

Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics),; Vol. 11206 LNCS(2018)p. 391-408

Paper in proceedings

Long-term 3D Localization and Pose from Semantic Labellings

IEEE International Conference on Computer Vision Workshops,; (2017)p. 650-659

Paper in proceedings

Toft, C., Sattler, T., Maddern, W., Torii, A., Hammarstrand, L., Stenborg, E., Safari, D., Okutomi, M., Pollefeys, M., Sivic, J., Pajdla, T., Kahl, F. Benchmarking 6DoF Outdoor Visual Localization in Changing Conditions

Subject Categories


Computer Science

Computer Vision and Robotics (Autonomous Systems)


Chalmers University of Technology


Opponent: Magnus Oskarsson, Lunds tekniska högskola, Sverige

More information

Latest update