Rethinking Code Review Workflows with LLM Assistance: An Empirical Study

Paper in proceeding, 2025

Background: Code reviews are a critical yet timeconsuming aspect of modern software development, increasingly challenged by growing system complexity and the demand for faster delivery.
Aims: We examine how large language models (LLMs) can support code reviews by addressing common inefficiencies and contextual gaps. Method: At WirelessCar Sweden A B, we conducted an exploratory field study to identify current challenges, followed by a field experiment with two LLM-assisted review prototypes: one providing upfront, AIgenerated reviews and another enabling on-demand interaction. Both used a retrieval-augmented generation pipeline to assemble relevant contextual information.
Results: The field study revealed frequent context switching, insufficient contextual information, and concerns around false positives. In practice, developers generally preferred the AI-led approach, especially for large or unfamiliar pull requests, though preferences varied with codebase familiarity and review risk.
Conclusions: LLM-assisted reviews can reduce cognitive load and improve comprehension, with hybrid proactive/on-demand designs best balancing efficiency, trust, and reviewer control.