Real-Time Dense Mapping for Self-Driving Vehicles using Fisheye Cameras
Paper in proceedings, 2019

We present a real-time dense geometric mapping algorithm for large-scale environments. Unlike existing methods which use pinhole cameras, our implementation is based on fisheye cameras whose large field of view benefits various computer vision applications for self-driving vehicles such as visual-inertial odometry, visual localization, and object detection. Our algorithm runs on in-vehicle PCs at approximately 15 Hz, enabling vision-only 3D scene perception for self-driving vehicles. For each synchronized set of images captured by multiple cameras, we first compute a depth map for a reference camera using plane-sweeping stereo. To maintain both accuracy and efficiency, while accounting for the fact that fisheye images have a lower angular resolution, we recover the depths using multiple image resolutions. We adopt the fast object detection framework, YOLOv3, to remove potentially dynamic objects. At the end of the pipeline, we fuse the fisheye depth images into the truncated signed distance function (TSDF) volume to obtain a 3D map. We evaluate our method on large-scale urban datasets, and results show that our method works well in complex dynamic environments.


Zhaopeng Cui

Swiss Federal Institute of Technology in Zürich (ETH)

Lionel Heng

DSO National Laboratories

Ye Chuan Yeo

DSO National Laboratories

Andreas Geiger

Max Planck Society

University of Tübingen

Marc Pollefeys

Swiss Federal Institute of Technology in Zürich (ETH)


Torsten Sattler

Chalmers, Electrical Engineering, Signalbehandling och medicinsk teknik, Imaging and Image Analysis


1050-4729 (ISSN) 2577-087X (eISSN)


International Conference on Robotics and Automation (ICRA)
Montreal, Canada,

Subject Categories

Media Engineering

Computer Vision and Robotics (Autonomous Systems)

Medical Image Processing

More information

Latest update