Relevant Safety Falsification by Automata Constrained Reinforcement Learning
Paper i proceeding, 2022

Complex safety-critical cyber-physical systems, such as autonomous cars or collaborative robots, are becoming increasingly common. Simulation-based falsification is a testing method for uncovering safety hazards of such systems already in the design phase. Conventionally, the falsification method takes the form of a static optimization. Recently, dynamic optimization methods such as reinforcement learning have gained interest for their ability to uncover harder-to-find safety hazards. However, these methods may converge to risk-maximising, but irrelevant behaviors. This paper proposes a principled formulation and solution of the falsification problem by automata constrained reinforcement learning, in which rewards for relevant behavior are tuned via Lagrangian relaxation. The challenges and proposed methods are demonstrated in a use-case example from the domain of industrial human-robot collaboration, where falsification is used to identify hazardous human worker behaviors that result in human-robot collisions. Compared to random sampling and conventional approximate Q-learning, we show that the proposed method generates equally hazardous, but at the same time more relevant testing conditions that expose safety flaws.

Författare

Constantin Cronrath

Chalmers, Elektroteknik, System- och reglerteknik

Tom P. Huck

Karlsruher Institut für Technologie (KIT)

Christoph Ledermann

Karlsruher Institut für Technologie (KIT)

Torsten Kroger

Karlsruher Institut für Technologie (KIT)

Bengt Lennartson

Chalmers, Elektroteknik, System- och reglerteknik

IEEE International Conference on Automation Science and Engineering

21618070 (ISSN) 21618089 (eISSN)

Vol. 2022-August 2273-2280
9781665490429 (ISBN)

18th IEEE International Conference on Automation Science and Engineering, CASE 2022
Mexico City, Mexico,

Ämneskategorier

Robotteknik och automation

Datavetenskap (datalogi)

Datorsystem

DOI

10.1109/CASE49997.2022.9926460

Mer information

Senast uppdaterat

2023-10-25