Calculus and Optimization

reading · we frame, you read MIT or the canonical taught · we author, no canonical fits ↺ spirals back to earlier lessons

Course locked

Complete Linear Algebra first.

This course unlocks once you've finished its prerequisite. Open prerequisite →

Prerequisites

Linear Algebra not yet

Complete prerequisites first to enroll.

reading · 35 min + ~290 min canonical

Derivatives and Gradients: The Direction of Steepest Change

If linear algebra is the language of data, calculus is the language of change and optimization — and here is the reassuring truth to carry into it: you need far less calculus for …

primary 3Blue1Brown — Essence of Calculus (YouTube)

preview →

reading · 35 min + ~290 min canonical L1

The Chain Rule and Backpropagation: Propagating Change Through a Pipeline

The last lesson said learning means following the negative gradient downhill — but it left a giant question unanswered: how do you actually compute the gradient of a model with mi…

primary 3Blue1Brown — Essence of Calculus (YouTube)

after L1 locked

reading · 35 min + ~290 min canonical L1

Optimization: Finding the Bottom of the Landscape

The previous lessons gave you the gradient (the downhill direction) and backpropagation (how to compute it). This lesson zooms out to the terrain itself: the loss landscape you're…

primary 3Blue1Brown — Essence of Calculus (YouTube)

after L2 locked

reading · 35 min + ~290 min canonical L1

Gradient Descent: How Machines Actually Learn

This is the lesson where everything converges into the single algorithm that trains essentially every machine-learning model in existence: gradient descent. You now have all the p…

primary 3Blue1Brown — Essence of Calculus (YouTube)

after L3 locked

reading · 35 min + ~290 min canonical L1

Beyond Vanilla: Constrained Optimization, Newton's Method, and Numerical Reality

The last lesson gave you gradient descent — the workhorse — but real optimization has three more dimensions the practitioner must understand, and this closing lesson covers them. …

primary 3Blue1Brown — Essence of Calculus (YouTube)

after L4 locked

exercise reading · 15 min L1

Exercises: Easy (Warm-up)

- [ ] Compute partial derivatives of f(x,y) = x^2y + sin(xy) — Multivariable calculus basics - [ ] Compute the gradient of f(x,y) = x^2 + 2y^2 - 4x + 6y — Find the minimum analyti…

after L5 locked

exercise reading · 30 min L1

Exercises: Medium (Core)

- [ ] Implement gradient descent for linear regression in Python — From scratch, no sklearn - [ ] Implement Newton's method for optimization in Python — Compare convergence speed …

after L6 locked

exercise reading · 45 min L1

Exercises: Hard (Stretch)

- [ ] Implement automatic differentiation (forward mode) in Python — Dual numbers, operator overloading - [ ] Solve constrained optimization using Lagrange multipliers — Maximize …

after L7 locked

exercise reading · 60 min L1

Mini-Project

- [ ] Build a mini-autograd engine in Python — Implement computation graph, forward pass, backward pass (chain rule), support +, , pow, relu, sigmoid; train a 2-layer MLP on synth…

after L8 locked

L10

project reading · 300 min L1

Project: Gradient Descent Visualizer

Implement gradient descent (batch, stochastic, mini-batch) for 2D functions in Python. Visualize the optimization path as a contour plot with matplotlib. Compare learning rates (t…

after L9 locked

L11

project reading · 240 min L1

Project: Curve Fitting with Gradient Descent

Fit a polynomial to noisy data using gradient descent. Implement the loss function (MSE), gradient computation (analytical), and optimization loop from scratch. Show underfitting …

after L10 locked

L12

assessment reading · 63 min L1

Self-Assessment: Calculus and Optimization

- [ ] What is a gradient? What does it represent geometrically? - [ ] Explain the chain rule. Why is it essential for backpropagation? - [ ] What is gradient descent? Explain the …

after L11 locked

12 lessons. Read in order; spiral back when you need to. By the end you'll have used the core ideas twice — once on the abstract, once on something you'll meet at work next week.