The robot as scientist: Using mental simulation to test causal hypotheses extracted from human activities in virtual reality
Paper i proceeding, 2020

To act effectively in its environment, a cognitive robot needs to understand the causal dependencies of all intermediate actions leading up to its goal. For example, the system has to infer that it is instrumental to open a cupboard door before trying to grasp an object inside the cupboard. In this paper, we introduce a novel learning method for extracting instrumental dependencies by following the scientific approach of observations, generation of causal hypotheses, and testing through experiments. Our method uses a virtual reality dataset containing observations from human activities to generate hypotheses about causal dependencies between actions. It detects pairs of actions with a high temporal co-occurrence and verifies if one action is instrumental in executing the other action through mental simulation in a virtual reality environment which represents the system's mental model. Our system is able to extract all present instrumental action dependencies while significantly reducing the search space for mental simulation, resulting in a 6-fold reduction in computational time.


Constantin Uhde

Technische Universität München

Nicolas Berberich

Technische Universität München

Karinne Ramirez-Amaro

Chalmers, Elektroteknik, System- och reglerteknik

Gordon Cheng

Technische Universität München

IEEE International Conference on Intelligent Robots and Systems

21530858 (ISSN) 21530866 (eISSN)

8081-8086 9341505
9781728162126 (ISBN)

IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)
Las Vegas, USA,


Människa-datorinteraktion (interaktionsdesign)

Robotteknik och automation

Datavetenskap (datalogi)



Mer information

Senast uppdaterat