Automated Exploration and Explanation of Software Boundaries

Licentiatavhandling, 2026

Boundary Value Analysis (BVA) is a software testing technique that targets inputs at the transitions between different program behaviors, where faults are most likely to occur. Automating this process, known as Boundary Value Exploration (BVE), requires not only finding input pairs that lie on opposite sides of these transitions, but finding enough of them, across different behavioral regions, to give testers a complete picture of where a program's behavior changes and why. This thesis advances BVE toward a search that is broader, more general across input types, and more interpretable to the testers.

To achieve this, this thesis focuses on unit-level functions and introduces a search-based framework that systematically seeks boundary candidates that are not only sharp but also spread across a wide range of behavioral regions, so that the search covers many different kinds of transitions rather than converging on the most extreme ones. Building on this foundation, a deeper limitation is addressed: existing approaches require search operators to be hand-crafted for each input type, confining BVE mostly to numeric domains. An agentic, LLM-driven framework that autonomously generates its own exploration strategies removes this bottleneck, extending BVE to functions with non-numeric inputs. Finally, since discovering boundaries is only part of the challenge, a mixed methods study with software professionals investigates whether LLMs can generate natural-language explanations for discovered boundaries, and what properties such explanations need to be used in practice.

The results show that a broader and more varied set of boundary behaviors can be discovered through diversity-aware search, and that adaptive, LLM-driven exploration generalizes successfully to input types that other existing automated black-box BVE approaches cannot handle. Software professionals found LLM-generated explanations useful overall, though the study also reveals that trust in such explanations is fragile and that correctness and consistency are prerequisites for adoption rather than desirable extras.

Together, these contributions suggest that the path toward practical BVE lies not in optimizing a single dimension of the search, but in balancing quality with diversity, and automation with human understanding. The findings point to diversity as an explicit and configurable testing goal, to LLMs as a practical mechanism for scaling BVE across input domains, and to explanation as a prerequisite for adoption rather than an optional enhancement.