AI Bootcamp V - Reinforcement Learning

Announcing the Fifth EAS AI Bootcamp: Reinforcement Learning

The fifth AI bootcamp will take place from June 10-14, 2024 in collaboration with Prof. Adam Wierman's group. During the bootcamp, you will dive into the fundamentals of Reinforcement Learning (RL), learn how to implement and optimize RL algorithms, and explore computational techniques that leverage these principles for solving complex decision-making and control problems. This course is designed  for Caltech graduate students and researchers and aims to accelerate their  research productivity through the practical applications of Reinforcement Learning. 

Objective:

This bootcamp will equip participants with an understanding of Reinforcement Learning (RL) fundamentals and techniques. Through a combination of theoretical lectures, hands-on coding exercises, and real-world case studies, researchers will develop the expertise to design and implement RL agents for decision-making tasks in their domains of expertise.

Key takeaways will include:

• Mastering core RL concepts such as Markov Decision Processes, value functions, and policy optimization.
• Exploring various RL algorithms (e.g., Q-learning, Policy Gradients, Actor-Critic methods) and their applications.
• Learning to design reward functions and optimize policies to achieve desired agent behaviors.
• Building and deploying RL agents in simulated and real-world environments.
• Gaining insights into current research trends and open challenges in the field of RL.

By the end of the bootcamp, participants will be prepared to:

• Apply RL principles to solve complex decision-making problems in their respective research domains.
• Develop innovative RL solutions tailored to specific tasks and environments.

Topics covered:

1. Fundamentals of RL: An introductory overview of RL, its significance, and real-world applications.
2. The Markov Decision Process (MDP) Framework / Dynamic Programming: In-depth discussion on states, actions, rewards, and the Markov property.  Formalizing the RL problem using MDPs. 
3. Model-Free Prediction  and Control Methods
4. Value Function Approximation
5. Policy Gradient Methods: Learning the Policy Directly
6. Integrating Learning and Planning: When to Act, When to Plan
7. Exploration vs. Exploitation –Bandit: The Balancing Act
8. Real-World Applications: Case Studies in Reinforcement Learning
9. (Tentative) Value Alignment: Ensuring Agents Behave As We Want

Prerequisites

To maximize your learning experience, familiarity with the following is required:

• Multivariable Calculus: Partial derivatives, integration, limits, and continuity.
• Probability Theory: Random variables, statistical measures, probability distributions, and Bayesian inference.
• Python Programming: Basic syntax and libraries
• PyTorch

In addition, you need to be familiar with ML concepts, specially linear models and Neural Networks and be comfortable training fairly complex models. You also need to be familiar with processing and analyzing data for use in ML. Please refer to the contents of the first bootcamp (use your Caltech credentials to login) to see the content that you need to be familiar with (or use the slides and hands-on practices there to make yourself familiar with the prerequisites) before registering. 

How to join:

• Bootcamp is limited  to 25  participants. If the number of requestors is more than this capacity we will chose the participants based on the following criteria: 
  • How well they do in the pre-screening Quiz
  • If they have sent us more details about their existing research and how they think this bootcamp can help them with their research. 
• Please sign up here and complete the pre-screening Quiz covering Python and ML prerequisites before the 11:59 PM Pacific Time on June 6th.  Please note that your enrollment won't be complete until you have taken the quiz and have received a confirmation email from the bootcamp organizers. For those who have already taken other bootcamp courses, you won't need to take the quiz, but please send an email to the organizers mentioning that you have already completed another AI bootcamp and therefore, have already taken the quiz. 
• (Highly recommended) email us about yourself and your research and let us know how you think that this bootcamp can help you with your research.  

Bootcamp Organizers: 

• Bootcamp director: Reza Sadri
• Administrative assistant: Caroline Murphy
• Instructors: Christopher Yeh, Zaiwei Chen, Laixi Shi, Kishan Panaganti Badrinath, Shi Zhuo Looi, Xiaozhou Fan
• TAs: Alejandro Stefan Zavala, Panteleimon Vafeidis, Agnim Agarwal

Working on your own AI projects during the bootcamp:

• Information about your project: We will create a Canvas submission page that allows you to submit information and background about your project (papers, web pages, background information, link to datasets you would like to use, etc). This allows us to tailor the program more towards the participants' needs. 
• Computing Resources: Most of the work for the hands-on sessions is going to be done using Google Colab. However, since we use the free version of Google Colab that provides only limited access to computing and memory resources, if you want to process larger datasets or train large models, you can buy Colab credits and get reimbursed after the bootcamp. Please contact Caroline Murphy for the details.

Deadline for registration: June 6, 2024