Home
Lectures
From Single-agent to Federated Reinforcement Learning

From Single-agent to Federated Reinforcement Learning

Speaker: Yuejie Chi – Pittsburgh, PA, United States
Topic(s): Artificial Intelligence, Machine Learning, Computer Vision, Natural language processing , Computational Theory, Algorithms and Mathematics

Abstract

Reinforcement learning (RL) is garnering significant interest in recent years due to its success in a wide variety of modern applications. Q-learning, which seeks to learn the optimal Q-function of a Markov decision process (MDP) in a model-free fashion, lies at the heart of RL practices. However, theoretical understandings on its non-asymptotic sample complexity remain unsatisfactory, despite significant recent efforts. In this talk, we first show a tight sample complexity bound of Q-learning in the single-agent setting, together with a matching lower bound to establish its minimax sub-optimality. We then show how federated versions of Q-learning allow collaborative learning using data collected by multiple agents without central sharing, where an importance averaging scheme is introduced to unveil the blessing of heterogeneity.

About this Lecture

Number of Slides: 40
Duration: 45 minutes
Languages Available: English
Last Updated:

Request this Lecture

To request this particular lecture, please complete this online form.

Request a Tour

To request a tour with this speaker, please complete this online form.

All requests will be sent to ACM headquarters for review.