Guided Gaussians: Enhancing 3D Occupancy Estimation with Sparse Sensor Priors

Paper in proceeding, 2025

We introduce a new initialization method for 3D Gaussians used in 3D occupancy estimation, a key task in autonomous driving that involves identifying semantic elements in a vehicle’s surroundings and accurately locating them in space. Our approach leverages distance sensor data, such as from lidar or radar, to place 3D Gaussians using farthest point sampling, ensuring coverage of meaningful scene areas while avoiding redundant representation of empty space. Unlike prior work that either densely voxelizes the scene or spreads 3D Gaussians uniformly, our method uses real sensor signals to drive object-centric placement, resulting in a more efficient and precise representation of the environment. We further enhance performance through a multimodal attention mechanism between 3D Gaussian features and distance sensor inputs, improving the integration of geometry and semantics. Our results show that this lightweight yet effective strategy consistently achieves state-of-the-art performance in 3D occupancy estimation. This contributes to a scalable solution for real-world deployment in autonomous vehicle perception systems, highlighting the potential of sensor-informed initialization for spatial reasoning in dynamic environments.