Welcome to CSCI 531 — Fall 2023

An introduction to reinforcement learning. Proficieny in python is required.

This course will introduce the fundamentals of Reinforcement learning (RL) and Deep learning techniques. The course will cover the Tabular solution methods, such as the finite Markov Decision Processes and Temporal-Difference learning. It will also cover approximation solution methods, as on-policy and off-policy approximations. By the end of the course, new deep-learning techniques will be introduced.

Professors

Dr. Jean-Alexis Delamer

• jdelamer at stfx.ca

• Annex 9C

Lecture Section

• Tue: 12:30pm - 1:20pm (MULH4024)

• Thu: 11:30am - 12:20pm (MULH4024)

• Fri: 1:30pm - 2:20pm (MULH4024)

Office Hours

• Tue: 1:30pm - 3:30pm (Annex 9C)

• Thu: 1:30pm - 2:30pm (Annex 9C)

• Fri: 9:00am - 10:00am (Annex 9C)

Notes

• 1. Introduction
• 2. Multi-armed bandit
• 3. Multi-armed bandit - Action Selection methods
• 4. Markov Decision Process
• 5. Policies and Value Function
• 6. Dynamic Programming
• 7. Monte Carlo Methods
• 8. Temporal-Difference Learning
• 9. Approximation - On-Policy
• 10. Eligibility Traces
• 11. Policy Gradient Methods

Labs

• 1. Multi-armed Bandit
• 2. Multi-armed bandit - Action Selection methods
• 3. Markov Decision Process
• 4. Dynamic Programming
• 5. Monte-Carlo Methods
• 6. TD Learning
• 7. Semi-gradient Sarsa
• 8. Eligibility-traces - Algorithms
• 9. Actor-Critic

Assignments

• 1. Spaceships rental
• 2. Maze Escape

Project

• Project

Outline

• Computer Science 531 - Reinforcement Learning