Decentralized Constrained Optimization: a Novel Convergence Analysis

Licentiate thesis, 2023

One reason for the spectacular success of machine learning models is the appearance of large datasets. These datasets are often generated by different computational units or agents and cannot be processed on a single machine due to memory and computing limitations. Moreover, the data may contain sensitive information and hence should not be shared among different machines. Distributed systems can handle these problems by keeping the data locally and leveraging the cooperation of agents over a communication graph. This thesis is focused on a family of distributed systems, where the objective is to minimize a sum of locally held functions subject to local constraints, called the Decentralized Constrained Optimization Problem (DCOP). This problem is of significant importance as it arises in various real-world applications such as distributed sensor networks, decentralized control, and multi-agent systems. Our main concern is to develop efficient first-order decentralized optimization algorithms to solve the DCOP and provide theoretical convergence guarantees.