Are we ready for beyond-application high-volume data? The Reeds robot perception benchmark dataset
Preprint, 2021

This paper presents a dataset, called Reeds, for research on robot perception algorithms. The dataset aims to provide demanding benchmark opportunities for algorithms, rather than providing an environment for testing application-specific solutions. A boat was selected as a logging platform in order to provide highly dynamic kinematics. The sensor package includes six high-performance vision sensors, two long-range lidars, radar, as well as GNSS and an IMU. The spatiotemporal resolution of sensors were maximized in order to provide large variations and flexibility in the data, offering evaluation at a large number of different resolution presets based on the resolution found in other datasets. Reeds also provides means of a fair and reproducible comparison of algorithms, by running all evaluations on a common server backend. As the dataset contains massive-scale data, the evaluation principle also serves as a way to avoid moving data unnecessarily.

It was also found that naive evaluation of algorithms, where each evaluation is computed sequentially, was not practical as the fetch and decode task of each frame would not scale well. Instead, each frame is only decoded once and then fed to all algorithms in parallel, including for GPU-based algorithms.

Author

Ola Benderius

Chalmers, Mechanics and Maritime Sciences, Vehicle Engineering and Autonomous Systems

Christian Berger

Chalmers, Computer Science and Engineering (Chalmers), Software Engineering (Chalmers), Software Engineering for Cyber Physical Systems

Krister Blanch

Chalmers, Mechanics and Maritime Sciences, Vehicle Engineering and Autonomous Systems

Referensdata och algoritmer till stöd för forskning och utveckling av smarta fartyg

Swedish Transport Administration (TRV2019/120103), 2020-04-15 -- 2023-03-31.

Subject Categories

Robotics

Computer Science

Computer Vision and Robotics (Autonomous Systems)

More information

Latest update

9/16/2021