Learning Robot Skills From Demonstration for Multi-Agent Planning

Paper in proceeding, 2024

Programming by Demonstration (PbD) is widely used to simplify the programming of new robot skills. The programmed skills can be sequenced using planning algorithms, enabling robots to perform new tasks. However, even though multi-robot systems are common in the industry, PbD methods have so far focused on single-robot scenarios. The direct deployment of PbD methods to multi-robot systems is not straightforward, since they do not consider the constraint of shared spaces and action synergies between robots. We, therefore, propose generally applicable guidelines for extending current PbD methods to Multi-Agent (MA) systems. We manually introduce a set of constrained movement skills into the skill library, which prevents the planner from scheduling two agents to act on a shared resource at the same time. Our guidelines also propose to use PbD method-specific knowledge, like action classification, to automatically detect and encode parallel actions, so they are only scheduled concurrently. We applied our proposed guidelines to kinesthetic teaching and demonstrations in Virtual Reality and assessed whether they allow non-experts to teach skills for MA tasks in a user study. Results show that our method can correctly encode the MA skill models, leading to 90% and 95% plan feasibility in a MA version of the Tower-of-Hanoi task with agents sharing workspace and an Opening/Closing-Bottle task requiring action synergies. In contrast, current non-MA methods result in plans that are unfeasible or are up to 40% less efficient in plan length.