Hands on Reinforcement
App Development
Mentors :
Ashwin Abraham
Jujhaar (200110052)
Prerak Contractor (210050124)
Sankalp Bhamare (200110096)"
Mentees :
10-12
This is a continuation of the WiDS projects "Hands-on Reinforcement Learning" and "Training AI to play games using Reinforcement Learning".
The repos for these projects are: https://github.com/AshwinAbraham2021/WiDS-Training-AI-to-play-games-using-Reinforcement-Learning and https://github.com/jujhaar2409/WiDS-Hands-on-RL
We will be following Sutton & Barto as a reference for the theory, and read chapters from it while implementing the strategies described in it in Python. For the Deep RL part, we will follow the book Grokking Deep Reinforcement Learning by Miguel Morales.
Some resources we may use are the slides on RL by David Silver (https://www.deepmind.com/learning-resources/introduction-to-reinforcement-learning-with-david-silver) and the slides from the course CS 747: Foundations of Intelligent and Learning Agents (https://www.cse.iitb.ac.in/~shivaram/teaching/old/cs747-a2022/index.html)
We will eventually train an RL model that can play (and win) simple games such as Tic-Tac-Toe Snake, and Atari Games. Eventually, we will train models on more complex games, including board games such as Chess.
Prerequisites:
Since this is a continuation of the WiDS projects, ideally, the applicants would have been mentees in one of these WiDS projects.
However, those with some preexisting Reinforcement Learning knowledge are also encouraged to apply (ideally you should've done the first 3 chapters of Sutton & Barto, although we may waive this requirement on an individual basis).
As always, enthusiasm is a prerequisite.
Tentative Timeline :
| Week | Work |
|---|---|
| Week 1 | S&B Chapter 4: Dynamic Programming Grokking Chapter 1: Introduction to Deep RL |
| Week 2 | S&B Chapter 5: Monte Carlo Methods Programming RL models to solve Tic-Tac-Toe and Snake |
| Week 3 | S&B Chapter 6: TD Learning and Q-Learning Grokking Chapter 8: Introduction to Value-based Deep RL |
| Week 4 | S&B Chapter 7: Eligibility Traces |
| Week 5 | S&B Chapter 8: Learning with Tabular Methods Grokking Chapters 9, 10: Deep Q-Learning Networks Coding up DQN models to play the Atari games in the Open AI gym (https://gymnasium.farama.org) |
| Week 6 | S&B Chapters 9, 11: Approximate Solution Methods Start coding up an RL based Chess Engine |
| Week 7 | S&B Chapter 14: Applications and Case Studies Grokking Chapters 11, 12: Advanced Methods |
