Rigorous Systems Research Group (RSRG) Seminar

Cyber-physical systems such as the power grid, Internet of Things (IoT), and transportation systems are increasingly adding large numbers of devices with sensing capabilities. This results in an explosion of data, and calls for a rethinking of traditional control/optimization theory, especially how to incorporate learning techniques to use the data for control/optimization purposes, as well as how to cope with the challenges caused by the sheer scale of the network. In the first part of the talk, we focus on distributed optimization, where the nodes seek to minimize a global loss function, which is a sum of their local loss functions formed by local data sets. This is a classical setting and our results provide the fastest known gradient-based distributed algorithm. In the second part, we go beyond static optimization and investigate how data can be directly used to design control policy. Particularly, we study reinforcement learning (RL) for localized control of networked systems. Despite its wide-ranging successes, the application of RL in the multi-agent systems has proven to be challenging due to scalability issues. Harnessing the network structure, we develop a Scalable Actor-Critic framework to learn an optimal local policy in a scalable manner. This result represents the first approach that provably addresses the issue of scalability in the context of multi-agent RL.