Keeping intelligence under control
Paper in proceedings, 2018

Modern software systems, such as smart systems, are based on a continuous interaction with the dynamic and partially unknown environment in which they are deployed. Classical development techniques, based on a complete description of how the system must behave in different environmental conditions, are no longer effective. On the contrary, modern techniques should be able to produce systems that autonomously learn how to behave in different environmental conditions.

Machine learning techniques allow creating systems that learn how to execute a set of actions to achieve a desired goal. When a change occurs, machine learning techniques allow the system to autonomously learn new policies and strategies for actions execution. This flexibility comes at a cost: the developer has no longer full control on the system behaviour. Thus, there is no way to guarantee that the system will not violate important properties, such as safety-critical properties.

To overcome this issue, we believe that machine learning techniques should be combined with suitable reasoning mechanisms aimed at assuring that the decisions taken by the machine learning algorithm do not violate safety-critical requirements. This paper proposes an approach that combines machine learning with run-time monitoring to detect violations of system invariants in the actions execution policies.

Machine learning

Safety-critical

Autonomous systems

Runtime verification

Reinforcement learning

Author

Piergiuseppe Mallozzi

Chalmers, Computer Science and Engineering (Chalmers), Software Engineering (Chalmers)

Patrizio Pelliccione

University of Gothenburg

Claudio Menghi

University of Gothenburg

Proceedings of the 1st International Workshop on Software Engineering for Cognitive Services

37-40

1st International Workshop on Software Engineering for Cognitive Services
Göteborg, Sweden,

Areas of Advance

Information and Communication Technology

Subject Categories

Embedded Systems

Computer Science

Computer Systems

DOI

10.1145/3195555.3195558

ISBN

9781450357401

More information

Latest update

3/18/2019