Long-Term Localization for Self-Driving Cars
Doktorsavhandling, 2020

Long-term localization is hard due to changing conditions, while relative localization within time sequences is much easier. To achieve long-term localization in a sequential setting, such as, for self-driving cars, relative localization should be used to the fullest extent, whenever possible.

This thesis presents solutions and insights both for long-term sequential visual localization, and localization using global navigational satellite systems (GNSS), that push us closer to the goal of accurate and reliable localization for self-driving cars. It addresses the question: How to achieve accurate and robust, yet cost-effective long-term localization for self-driving cars?

Starting in this question, the thesis explores how existing sensor suites for advanced driver-assistance systems (ADAS) can be used most efficiently, and how landmarks in maps can be recognized and used for localization even after severe changes in appearance. The findings show that:
* State-of-the-art ADAS sensors are insufficient to meet the requirements for localization of a self-driving car in less than ideal conditions.
GNSS and visual localization are identified as areas to improve. 
* Highly accurate relative localization with no convergence delay is possible by using time relative GNSS observations with a single band receiver, and no base stations. 
* Sequential semantic localization is identified as a promising focus point for further research based on a benchmark study comparing state-of-the-art visual localization methods in challenging autonomous driving scenarios including day-to-night and seasonal changes. 
* A novel sequential semantic localization algorithm improves accuracy while significantly reducing map size compared to traditional methods based on matching of local image features. 
* Improvements for semantic segmentation in challenging conditions can be made efficiently by automatically generating pixel correspondences between images from a multitude of conditions and enforcing a consistency constraint during training. 
* A segmentation algorithm with automatically defined and more fine-grained classes improves localization performance. 
* The performance advantage seen in single image localization for modern local image features, when compared to traditional ones, is all but erased when considering sequential data with odometry, thus, encouraging to focus future research more on sequential localization, rather than pure single image localization.



Sequential semantic localization

Opponent: Jan-Michael Frahm, University of North Carolina at Chapel Hill, USA


Erik Stenborg

Chalmers, Elektroteknik, Signalbehandling och medicinsk teknik, Signalbehandling

Vehicle self-localization using off-the-shelf sensors and a detailed map

IEEE Intelligent Vehicles Symposium, Proceedings,; (2014)p. 522-528

Paper i proceeding

Using a single band GNSS receiver to improve relative positioning in autonomous cars

IEEE Intelligent Vehicles Symposium, Proceedings,; Vol. 2016-August(2016)p. Art no 7535498, Pages 921-926

Paper i proceeding

C. Toft, W. Maddern, A. Torii, L. Hammarstrand, E. Stenborg, D. Safari, M. Okutomi, M. Pollefeys, J. Sivic, T. Pajdla, F. Kahl, and T. Sattler, Long-Term Visual Localization Revisited

E. Stenborg, C. Toft, and L. Hammarstrand, Semantic Maps and Long-Term Self-Localization for Self-Driving Cars using Image Segmentation

A cross-season correspondence dataset for robust semantic segmentation

Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition,; Vol. 2019-June(2019)p. 9524-9534

Paper i proceeding

Fine-Grained Segmentation Networks: Self-Supervised Segmentation for Improved Long-Term Visual Localization

Proceedings of the IEEE International Conference on Computer Vision,; (2019)p. 31-41

Paper i proceeding

E. Stenborg, T. Sattler, and L. Hammarstrand, Using Image Sequences for Long-Term Visual Localization

Imagine being blindfolded and transported to some random place and then opening your eyes for just one second. Unless you are in a very familiar place, it would be very hard to directly understand where you are. However, if you are let free to explore the surroundings with open eyes, you have no trouble to relate your new location as compared to where you were one second before. After a while, the leads to where you are begin to pile up, and you can usually figure out your location in the world. Likewise, a self-driving car accumulates data over time in order to solve the problem of localizing itself in the world.

The research in this thesis aims at developing algorithms that self-driving cars can use to find their location in the world. The thesis presents solutions and insights both for long-term sequential visual localization, and localization using GPS, that push us closer to the goal of accurate and reliable localization for self-driving cars. Visual localization, that is, figuring out where you are from images, is proposed to be based on semantic segmentation. It means that the images are interpreted by a computer into class labels such as "building", "road", "traffic sign", for each pixel in the image, and using this to infer location. A localization solution for the case when the camera on the car delivers images in a sequence, is presented and compared to alternative ways of localization. Two methods for improved semantic segmentation are also presented, in turn leading to better localization performance. Additionally, a method for localization using common car sensors (including radars, camera, and GPS receivers), and a method for improving the relative localization performance of GPS, are presented.




C3SE (Chalmers Centre for Computational Science and Engineering)


Datorseende och robotik (autonoma system)



Doktorsavhandlingar vid Chalmers tekniska högskola. Ny serie: 4844


Chalmers tekniska högskola


Opponent: Jan-Michael Frahm, University of North Carolina at Chapel Hill, USA

Mer information

Senast uppdaterat