Course · 12 lessons ~42 hr Advanced

Reinforcement Learning

Reinforcement Learning covers: The Reinforcement Learning Problem and Markov Decision Processes, Value Functions and Dynamic Programming, Learning from Experience, Exploration, Function Approximation, and Deep Reinforcement Learning, Policy Gradients and Reinforcement Learning in the Real World. Year 4, Quarter 14. Includes 11 exercises and 1 projects.

reading · we frame, you read MIT or the canonical taught · we author, no canonical fits ↺ spirals back to earlier lessons

Course locked

Complete Unsupervised Learning and Practical ML first.

This course unlocks once you've finished its prerequisite. Open prerequisite →

Prerequisites

Probability and Statistics not yet

Supervised Learning not yet

Complete prerequisites first to enroll.

reading · 35 min + ~210 min canonical

The Reinforcement Learning Problem and Markov Decision Processes

Supervised learning had labels. Unsupervised learning had structure. Reinforcement learning has neither — it has a goal and trial and error. This is the third and most different p…

primary David Silver — Reinforcement Learning Course (DeepMind, YouTube)

preview →

reading · 35 min + ~210 min canonical L1

Value Functions and Dynamic Programming: The Bellman Equation

The last lesson posed RL's central difficulty: credit assignment — when a reward arrives late, which earlier actions deserve the credit? This lesson introduces the idea that solve…

primary David Silver — Reinforcement Learning Course (DeepMind, YouTube)

after L1 locked

reading · 35 min + ~210 min canonical L1

Learning from Experience: Monte Carlo and Temporal Difference

Dynamic programming (last lesson) solves RL exactly — but only if you have a perfect model of the environment. Real agents don't. A robot doesn't have equations for the world; a g…

primary David Silver — Reinforcement Learning Course (DeepMind, YouTube)

after L2 locked

reading · 35 min + ~210 min canonical L1

Exploration, Function Approximation, and Deep Reinforcement Learning

Two problems stand between the tabular Q-learning of the last lesson and RL that works on real problems — and solving them is what this lesson is about. The first is a dilemma so …

primary David Silver — Reinforcement Learning Course (DeepMind, YouTube)

after L3 locked

reading · 40 min + ~210 min canonical L1

Policy Gradients and Reinforcement Learning in the Real World

Every RL method so far has learned a value function and then derived a policy from it ("pick the highest-value action"). This lesson introduces the alternative that powers much of…

primary David Silver — Reinforcement Learning Course (DeepMind, YouTube)

after L4 locked

exercise reading · 15 min L1

Exercises: Easy (Warm-up)

- [ ] Implement value iteration for a gridworld MDP in Python — 4x4 grid, terminal states, discount factor - [ ] Implement policy evaluation — Iteratively compute V(s) for a given…

after L5 locked

exercise reading · 30 min L1

Exercises: Medium (Core)

- [ ] Implement Q-Learning for FrozenLake — OpenAI Gym environment, Q-table, train to solve 4x4 and 8x8 - [ ] Implement SARSA and compare with Q-Learning — On-policy vs off-policy…

after L6 locked

exercise reading · 45 min L1

Exercises: Hard (Stretch)

- [ ] Implement Deep Q-Network (DQN) in PyTorch — Replay buffer, target network, train on CartPole - [ ] Implement REINFORCE (policy gradient) in PyTorch — Monte Carlo policy grad…

after L7 locked

exercise reading · 60 min L1

Mini-Project

- [ ] Build a tic-tac-toe AI using RL in Python — Q-learning agent that learns optimal play through self-play; train for 100K games, evaluate against random opponent, then minimax…

after L8 locked

L10

project reading · 360 min L1

Project: Grid World RL Agent

Implement value iteration, Q-learning, and SARSA for a configurable grid world (walls, traps, goals with different rewards). Visualize the learned policy (arrows showing optimal a…

after L9 locked

L11

assessment reading · 56 min L1

Self-Assessment: Reinforcement Learning

- [ ] What is a Markov Decision Process? Explain states, actions, transitions, and rewards. - [ ] What is the difference between value-based and policy-based methods? - [ ] Explai…

after L10 locked

L12

assessment reading · 60 min L1

Quarter 14 Gate Check: Classical Machine Learning

- [ ] Course 14.1 knowledge check passed - [ ] Course 14.2 knowledge check passed - [ ] Course 14.3 knowledge check passed - [ ] Kaggle Competition capstone completed

after L11 locked

12 lessons. Read in order; spiral back when you need to. By the end you'll have used the core ideas twice — once on the abstract, once on something you'll meet at work next week.