Neural Networks and Deep Learning Fundamentals

reading · we frame, you read MIT or the canonical taught · we author, no canonical fits ↺ spirals back to earlier lessons

Course locked

Complete Reinforcement Learning first.

This course unlocks once you've finished its prerequisite. Open prerequisite →

Prerequisites

Calculus and Optimization not yet

Supervised Learning not yet

Complete prerequisites first to enroll.

reading · 35 min + ~258 min canonical

From Neuron to Network: The Multi-Layer Perceptron

This is where AI stops being "statistics with computers" and becomes something genuinely new. A neural network does something the classical models of the last quarter could not: i…

primary Andrej Karpathy — Neural Networks: Zero to Hero (YouTube)

preview →

reading · 40 min + ~258 min canonical L1

Backpropagation and Computation Graphs

The last lesson built a network and ran the forward pass — but a fresh network has random weights and predicts garbage. Learning means adjusting those weights (often millions or b…

primary Andrej Karpathy — Neural Networks: Zero to Hero (YouTube)

after L1 locked

reading · 40 min + ~258 min canonical L1

Making Deep Networks Train: Initialization, Optimizers, and Normalization

The last lesson ended on a warning: deep networks are hard to train, because gradients vanish or explode as they propagate back through many layers. This lesson is the toolkit tha…

primary Andrej Karpathy — Neural Networks: Zero to Hero (YouTube)

after L2 locked

reading · 35 min + ~258 min canonical L1

Convolutional Neural Networks: Vision and Spatial Structure

The classical-ML course drew a hard line: tabular data → boosted trees; unstructured data (images, text, audio) → deep learning. This lesson is the first payoff of the "unstructur…

primary Andrej Karpathy — Neural Networks: Zero to Hero (YouTube)

after L3 locked

reading · 35 min + ~258 min canonical L1

Transfer Learning, and the Opacity of Deep Networks

Two things close out the deep-learning fundamentals — one intensely practical, one deeply honest. The practical one is transfer learning, and it's the technique that makes deep le…

primary Andrej Karpathy — Neural Networks: Zero to Hero (YouTube)

after L4 locked

exercise reading · 15 min L1

Exercises: Easy (Warm-up)

- [ ] Karpathy Lecture 1: Build micrograd — Implement scalar-valued autograd engine, train a tiny neural net from scratch - [ ] Implement a perceptron in Python — AND, OR, XOR gat…

after L5 locked

exercise reading · 30 min L1

Exercises: Medium (Core)

- [ ] Karpathy Lecture 2: Build makemore Part 1 — Bigram character-level language model with counting - [ ] Karpathy Lecture 3: Build makemore Part 2 — MLP language model, train/d…

after L6 locked

exercise reading · 45 min L1

Exercises: Hard (Stretch)

- [ ] Karpathy Lecture 6: Build makemore Part 5 — WaveNet-like architecture with dilated causal convolutions - [ ] Implement a CNN from scratch in PyTorch — Convolution, pooling, …

after L7 locked

exercise reading · 60 min L1

Mini-Project

- [ ] Build a digit classifier — Train a CNN on MNIST from scratch in PyTorch: convolutional layers, batch norm, dropout, Adam optimizer; achieve >99% accuracy; deploy as a Flask …

after L8 locked

L10

project reading · 360 min L1

Project: Micrograd in Python

Follow Karpathy's lecture 1 and build a scalar-valued autograd engine from scratch. Then extend it: add tanh, exp, and log operations. Train a small MLP on a binary classification…

after L9 locked

L11

project reading · 480 min L1

Project: Makemore (Parts 1-3)

Follow Karpathy's lectures 2-4. Build a character-level language model: bigram (counting), then MLP, then with proper BatchNorm and initialization. Generate names from each model …

after L10 locked

L12

project reading · 600 min L1

Capstone: Train and Deploy a Neural Network

- [ ] Milestone 1: Train nanoGPT on a custom text corpus (>1MB of text) - [ ] Milestone 2: Implement temperature, top-k, and top-p sampling for generation - [ ] Milestone 3: Train…

after L11 locked

L13

assessment reading · 63 min L1

Self-Assessment: Neural Networks and Deep Learning Fundamentals

- [ ] Explain backpropagation. How does the chain rule enable gradient computation through layers? - [ ] What are vanishing and exploding gradients? How do ReLU and batch normaliz…

after L12 locked

13 lessons. Read in order; spiral back when you need to. By the end you'll have used the core ideas twice — once on the abstract, once on something you'll meet at work next week.