Real-Time Dense Mapping for Self-Driving Vehicles using Fisheye Cameras
Paper i proceeding, 2019

We present a real-time dense geometric mapping algorithm for large-scale environments. Unlike existing methods which use pinhole cameras, our implementation is based on fisheye cameras whose large field of view benefits various computer vision applications for self-driving vehicles such as visual-inertial odometry, visual localization, and object detection. Our algorithm runs on in-vehicle PCs at approximately 15 Hz, enabling vision-only 3D scene perception for self-driving vehicles. For each synchronized set of images captured by multiple cameras, we first compute a depth map for a reference camera using plane-sweeping stereo. To maintain both accuracy and efficiency, while accounting for the fact that fisheye images have a lower angular resolution, we recover the depths using multiple image resolutions. We adopt the fast object detection framework, YOLOv3, to remove potentially dynamic objects. At the end of the pipeline, we fuse the fisheye depth images into the truncated signed distance function (TSDF) volume to obtain a 3D map. We evaluate our method on large-scale urban datasets, and results show that our method works well in complex dynamic environments.


Zhaopeng Cui

Eidgenössische Technische Hochschule Zürich (ETH)

Lionel Heng

DSO National Laboratories

Ye Chuan Yeo

DSO National Laboratories

Andreas Geiger

Universität Tübingen


Marc Pollefeys

Eidgenössische Technische Hochschule Zürich (ETH)


Torsten Sattler

Chalmers, Elektroteknik, Signalbehandling och medicinsk teknik

Proceedings - IEEE International Conference on Robotics and Automation

10504729 (ISSN)

978-1-5386-6026-3 (ISBN)

International Conference on Robotics and Automation (ICRA)
Montreal, Canada,



Datorseende och robotik (autonoma system)

Medicinsk bildbehandling





Mer information

Senast uppdaterat