Evaluating Two Semantics for Falsiﬁcation using an Autonomous Driving Example
Proceedings (editor), 2019
We consider the falsiﬁcation of temporal logic properties as a method to test complex systems, such as autonomous systems. Since these systems are often safety-critical, it is important to assess whether they fulﬁll given speciﬁcations or not. An adaptive cruise controller for an autonomous car is considered where the closed-loop model has unknown parameters and an important problem is to ﬁnd parameter combinations for which given speciﬁcation are broken. We assume that the closed-loop system can be simulated with the known given parameters, no other information is available to the testing framework. The speciﬁcation, such as, the ability to avoid collisions, is expressed using Signal Temporal Logic (STL). In general, systems consist of a large number of parameters, and it is not possible or feasible to explicitly enumerate all combinations of the parameters. Thus, an optimization-based approach is used to guide the search for parameters that might falsify the speciﬁcation. However, a key challenge is how to select the objective function such that the falsiﬁcation of the speciﬁcation, if it can be falsiﬁed, can be falsiﬁed using as few simulations as possible. For falsiﬁcation using optimization it is required to have a measure representing the distance to the falsiﬁcation of the speciﬁcation. The way the measure is deﬁned results in different objective functions usedduringoptimization.Differentmeasureshavebeenproposed in the literature and in this paper the properties of the Max Semantics (MAX) and the Mean Alternative Robustness Value (MARV) semantics are discussed. After evaluating these two semantics on an adaptive cruise control example, we discuss their strengths and weaknesses to better understand the properties of the two semantics.
Mean Alternative Robustness Value