Neural Rendering for Autonomous Driving

Doctoral thesis, 2026

Autonomous driving holds the potential to fundamentally transform transportation, but realizing that potential requires rigorous testing across a vast and diverse range of scenarios. Exhaustive real-world testing is neither safe for exploring edge cases nor practical at the scale required for adequate coverage, making high-fidelity simulation an essential component of the development and validation pipeline. This thesis addresses the challenge of building digital twins for autonomous driving: data-driven virtual replicas of real-world scenes that faithfully reproduce sensor observations while remaining editable for counterfactual testing.
The first contribution is NeuRAD, a neural simulator that jointly reconstructs camera and lidar data from recorded driving sequences. By explicitly modeling key sensor characteristics, including rolling shutter effects, beam divergence, and non-returning lidar rays, NeuRAD achieves state-of-the-art rendering fidelity across multiple autonomous driving datasets.
The second contribution is SplatAD, which replaces the volumetric representation of NeuRAD with 3D Gaussian primitives and purpose-built rasterization algorithms. This yields real-time rendering of both camera and lidar data at competitive fidelity, while reducing training and inference times by an order of magnitude, making large-scale simulation substantially more practical.
The third contribution is IDSplat, which eliminates the reliance on human-annotated 3D bounding boxes required by both NeuRAD and SplatAD. By combining vision foundation models with classical matching and estimation techniques, IDSplat achieves annotation-free dynamic scene reconstruction that generalizes zero-shot to new datasets.
The fourth contribution is a highly parallelizable GPU algorithm for constructing large-scale 3D Voronoi and power diagrams, addressing a key computational bottleneck in mesh-based neural rendering and enabling larger, higher-fidelity scene representations.
Finally, the thesis investigates the real-to-sim gap: the discrepancy between how autonomous systems perceive real and rendered sensor data. Through large-scale evaluation, correlations between rendering quality and downstream perception performance are identified, and fine-tuning strategies are developed that improve model robustness to simulation artifacts without compromising real-world performance.
Together, these contributions advance the state of neural simulation for autonomous driving, bringing scalable, high-fidelity virtual testing closer to practical deployment.