Deep Reinforcement Learning for Multiple Agents in a Decentralized Architecture: A Case Study in the Telecommunication Domain
Paper in proceeding, 2023

Deep reinforcement learning has made significant development in recent years, and it is currently applied not only in simulators and games but also in embedded systems. However, when implemented in a real-world context, reinforcement learning is frequently shown to be unstable and incapable of adapting to realistic situations, particularly when directing a large number of agents. In this paper, we develop a decentralized architecture for reinforcement learning to allow multiple agents to learn optimal control policies on their own devices of the same kind but in varied environments. For such multiple agents, the traditional centralized learning algorithm usually requires a costly or time-consuming effort to develop the best-regulating policy and is incapable of scaling to a large-scale system. To address this issue, we propose a decentralized reinforcement learning algorithm (DecRL) and information exchange scheme for each individual device, in which each agent shares the individual learning experience and information with other agents based on local model training. We incorporate the algorithm into each agent in the proposed collaborative architecture and validate it in the telecommunication domain under emergency conditions, in which a macro base station (BS) is broken due to a natural disaster, and three unmanned aerial vehicles carrying BSs (UAV-BSs) are deployed to provide temporary coverage for mission-critical (MC) users in the disaster area. Based on the findings, we show that the proposed decentralized reinforcement learning algorithm can successfully support multi-agent learning, while the learning speed and service quality can be further enhanced.

Reinforcement Learning

Software Engineering

Machine Learning

Decentralized Architecture

Multi-Agent

Emergency Communication Network

Author

Hongyi Zhang

Chalmers, Computer Science and Engineering (Chalmers), Interaction Design and Software Engineering

Jingya Li

Ericsson

Zhiqiang Qi

Ericsson

Anders Aronsson

Ericsson

Jan Bosch

Chalmers, Computer Science and Engineering (Chalmers), Interaction Design and Software Engineering

Helena Holmström Olsson

Malmö university

Proceedings - IEEE 20th International Conference on Software Architecture Companion, ICSA-C 2023

183-186
9781665464598 (ISBN)

20th IEEE International Conference on Software Architecture Companion, ICSA-C 2023
L'Aquila, Italy,

Subject Categories

Telecommunications

Software Engineering

Robotics

Computer Science

DOI

10.1109/ICSA-C57050.2023.00048

More information

Latest update

5/26/2023