Vectors, Matrices, and the Geometry of Data

Hook #

You are beginning the math that powers machine learning, and here is the single most liberating fact to carry into it: linear algebra is the most visual branch of mathematics — every concept has a picture you can draw — and it is the math that matters most for ML, so you are learning the highest-leverage, most-intuitive material first. This is not the abstract symbol-manipulation of number theory; a vector is an arrow you can see, a matrix is a transformation that moves space, the dot product measures how much two directions agree. And the reframe that turns linear algebra from "school math you half-remember" into "the language of data" is this: a piece of data — a user, an image, a sentence, a house for sale — is a vector, a point in a space of features, and everything machine learning does is geometry in that space. A recommendation engine finds vectors that point the same way; a classifier draws a boundary that separates clusters of points; a neural network is a stack of matrix transformations bending that space until the data becomes separable. Once you see data as geometry, machine learning stops being a bag of mysterious tricks and becomes something you can picture. This first lesson builds that foundation — vectors, matrices, the dot product, and norms — and, crucially, the habit of seeing the geometry, because that habit is what makes the rest of ML comprehensible.

What you'll be able to do by the end of this lesson #

• Think of a vector as a data point — a list of feature values that is simultaneously an arrow in a high-dimensional space — and see why "data lives in a vector space" is the foundational reframe of ML.
• Use the dot product as the measure of similarity/alignment between vectors (and recognize cosine similarity as its normalized form) — the operation behind search, recommendations, and every neural-network layer.
• Hold the dual nature of a matrix: it is both a transformation of space and a container of data (a dataset is a matrix of vectors; a transformation is a matrix too) — and why this duality is the heart of linear algebra's power.
• Use vector norms (L1/Manhattan, L2/Euclidean, Frobenius for matrices) to measure the size of a vector or the distance between data points, and know why the choice of norm matters.

A quick try before we start #

Represent three movies as vectors of [action, romance, comedy] scores: an action flick [9, 1, 2], a rom-com [1, 9, 6], and another action movie [8, 2, 1]. Which two are most "similar"? Your intuition says the two action movies — and the dot product proves it: the two action vectors point in nearly the same direction (their dot product is large), while the rom-com points elsewhere (its dot product with either action movie is smaller). You just did the core operation of a recommendation engine: represent items as vectors of features, and measure similarity as alignment via the dot product. Notice three things. First, you turned "movies" into geometry — they're now points in a 3D "taste space." Second, "similar" became a precise, computable thing (dot product / cosine similarity), not a vague feeling. Third, this scales: real systems use vectors with thousands of dimensions (every word, every user behavior, every pixel), but the operation is identical — alignment via dot product. Seeing that data becomes geometry and similarity becomes a dot product is the entire foundation of this course, and you just used it.

Why this matters here #

This lesson opens Linear Algebra — the first course of Year 4, the mathematics-for-ML year — and it establishes the reframe the whole year depends on: data is geometry, and linear algebra is the language for reasoning about it. Every ML topic ahead is built on this. A dataset is a matrix (rows of feature vectors); a linear model is a dot product (weights dotted with features); a neural network is a sequence of matrix transformations; dimensionality reduction (PCA, coming later in this course) is finding the directions of greatest variance in the data's geometry; embeddings (words, images) are vectors whose dot products encode meaning. So this course is not a detour into pure math — it's the acquisition of the vocabulary and intuition that makes all of ML sayable and seeable. This lesson also connects directly backward: the graphics course (12.3) already introduced vectors, the dot product, and matrices-as-transformations in the visual setting of rendering — this course takes that same math and points it at data, which is a beautiful continuity (the linear algebra that rotates a teapot is the linear algebra that classifies an image), and a reason the graphics elective was such good preparation.

For a working engineer approaching ML, this material is the difference between understanding your models and merely running them — and that difference is exactly where real ML capability lives. The honest truth about ML math is that you need less than gatekeepers claim but more than "just call the library" bootcamps admit: you need enough to understand why a model fails, not just that it failed. When a model's training diverges, you need to know what a gradient is (calculus, next course) and what the weight matrix is doing (this course). When embeddings "capture meaning," you need to see that meaning is geometric (similar things are nearby vectors). When a recommendation is bad, you need to reason about the vector space it's operating in. The engineer who has internalized "data is geometry" can debug, reason about, and improve ML systems; the one who hasn't can only try random hyperparameters and hope. And there's a genuinely encouraging aspect the whole year hinges on: linear algebra is the friendliest of the ML maths — it's visual, it's intuitive, every concept has a picture, and for a programmer it connects to things you already know (a matrix multiply is a nested loop; a dataset is a 2D array; a transformation is a function). This course is where you build the geometric intuition that makes the harder maths (calculus, probability) and the ML itself far more approachable. Watching the geometry — literally picturing the vectors and transformations — is the single most valuable habit you can build here, and it starts now.

The engineer's lens #

The first lens is the vector as data — the reframe that a data point is a point in a feature space, which turns the abstract machinery of linear algebra into the concrete language of ML. A vector is, mechanically, just an ordered list of numbers — [9, 1, 2] — which a programmer recognizes instantly as an array. But the insight is that this list is simultaneously two things: a data point (three feature values describing a movie) and an arrow in space (a point in 3D you could plot). This dual reading is the foundation of everything. When you represent a user as a vector of their behaviors, an image as a vector of its pixels, a word as a learned embedding vector, or a house as a vector of [bedrooms, sqft, price, ...], you have placed that entity in a high-dimensional feature space where geometric relationships encode real relationships: similar entities are nearby points, dissimilar ones are far apart, and the directions in the space often correspond to meaningful axes of variation. This is why ML is, at bottom, geometry on data: classification is drawing boundaries between regions of the space, clustering is finding groups of nearby points, recommendation is finding aligned vectors, and learning is reshaping the space so the data becomes easy to separate. The reason this reframe is so powerful for an engineer is that geometry is picturable and reasoning about pictures is easier than reasoning about equations — "these two users are similar" becomes "their vectors point the same way," "this feature doesn't matter" becomes "the data doesn't vary along that axis," "the model can't separate these classes" becomes "no line divides these clusters." You will spend all of Year 4 building on this single move (entity → feature vector → point in space), so internalizing it now — a data point is a vector is a point in a space where geometry means something — is the highest-leverage thing in the lesson. And it connects to what you already know: a dataset is then a matrix (a stack of these row vectors), which is exactly the 2D array you've manipulated as a programmer forever — you've been handling vectors and matrices all along; this course gives you the geometric eyes to see what they mean.

The second lens is the dot product as the universal similarity operation — the single most important computation in ML — because "how aligned are two vectors?" is the question underneath search, recommendation, and every layer of a neural network. The dot product takes two vectors and returns a single number that measures how much they point in the same direction: large and positive when aligned, zero when perpendicular, negative when opposed (it's the sum of element-wise products, and geometrically it's the product of the vectors' lengths and the cosine of the angle between them). You met it in the graphics course as the heart of lighting (a surface's brightness is the alignment of its normal with the light); here it's the heart of meaning. Cosine similarity — the dot product normalized by the vectors' lengths, isolating pure direction — is the standard measure of similarity in ML and information retrieval: semantic search finds documents whose embedding vectors align with the query's; recommendation finds items whose vectors align with your taste vector; near-duplicate detection finds vectors pointing the same way. And it goes deeper: a linear model (the foundation of ML) computes its prediction as a dot product of a learned weight vector with the input's feature vector — the weights say "how much does each feature matter, and in which direction," and the dot product combines them into a score. A neural network layer is, at its core, a matrix of dot products (each output neuron dots the inputs with its own weight vector). So the dot product is not one operation among many — it is the operation, repeated billions of times, that ML is built from, which is why the GPU (from the graphics course) exists: it's a machine for doing enormous numbers of dot products (matrix multiplications) in parallel, and that's why it powers deep learning. Internalizing the dot product as "the alignment/similarity question, answered with one number" — and recognizing it wherever "how similar?" or "combine these features into a score" appears — is the second foundational move of the course, and it's the operation you'll see more than any other in all of ML.

The third lens is the dual nature of the matrix — it is both a transformation of space and a container of data — and the vector norms that let you measure size and distance, because these are the tools that turn "data is geometry" into something you can compute with. A matrix wears two hats, and holding both is key to linear algebra's power. As a transformation, a matrix (multiplied by a vector) moves that vector — rotating, scaling, shearing, projecting it (the graphics-course reading of a matrix: it does something to space) — and this is how ML reshapes data: a neural network's weight matrices transform the feature space, layer by layer, bending it until the data becomes linearly separable, so "learning" is largely "finding the right transformations." As a container of data, a matrix is simply a stack of vectors — a dataset is an m × n matrix (m examples, each an n-dimensional feature vector), an image is a matrix of pixels, a batch of embeddings is a matrix — so the same object (a rectangular grid of numbers) is both the data you operate on and the operations you apply. This duality is not a coincidence to memorize but the deep structural fact that makes linear algebra a unified language: transformations and data are the same kind of thing, which is why you can multiply them together freely (transform a whole dataset at once by multiplying two matrices), and why the entire computation of a neural network is just matrix multiplications (data matrices times weight matrices). Alongside this, norms give you the ruler: a norm measures the size (length) of a vector, and thus the distance between two data points (the norm of their difference). The L2 (Euclidean) norm is the familiar straight-line length (√ of sum of squares) — the "as the crow flies" distance, the default for most geometry. The L1 (Manhattan) norm is the sum of absolute values — "city-block" distance, and important in ML because it induces sparsity (L1 regularization drives weights to exactly zero, a feature-selection effect you'll meet later). The Frobenius norm extends this to matrices (the L2 norm treating the matrix as one long vector). Norms matter because ML is full of "how big?" and "how far?" questions — how large is this weight (regularization penalizes big weights), how far is this prediction from the truth (the L2 norm of the error is the loss), how close are these two data points (distance-based methods like k-nearest-neighbors) — and the choice of norm changes the answer and the behavior (L1 vs L2 give different regularization and different notions of distance). Together, the matrix's duality (transformation and data) and the norm (size and distance) are the computational tools that make "data is geometry" operational — you can now transform data, measure it, and compute distances in it, which is the machinery all of ML runs on. The synthesis of the whole lesson: represent entities as vectors (points in feature space), measure their similarity with the dot product and their distance with a norm, and reshape the space with matrix transformations — that quartet is the geometric foundation of machine learning, and everything ahead builds on it.

What to focus on in the resources #

• 3Blue1Brown's Essence of Linear Algebra — primary, watch it FIRST. This is the highest-ROI resource in the course. It builds the geometric intuition — vector, matrix-as-transformation, dot-product-as-projection — that makes every textbook chapter afterward dramatically easier. Watch the whole series once before reading anything.
• Deisenroth (MML) Ch 2-3, free. Read for the ML-focused rigor on vectors, matrices, dot products, and norms. This is the canonical text the whole Year-4 math sequence is built on; the free PDF and notebooks are all you need.
• Strang's MIT 18.06 (early lectures), free. Watch for a deeper, geometry-first treatment and Strang's insistence on seeing the math — the right posture for the whole course.
• Skip on first pass: the eight vector-space axioms (formal but not yet needed — the geometry matters more than the formalism now), and exhaustive matrix-operation drills. Get: vector-as-data-point, dot-product-as-similarity (and cosine similarity), matrix's dual nature (transformation and data), and norms as size/distance. Above all, build the habit of picturing the geometry.

Explain it back #

Explain to a colleague why "data is geometry" is the foundational idea of ML, and why the dot product is the single most important operation. A strong answer: any entity — a user, image, word, house — can be represented as a vector, an ordered list of feature values that is also a point (an arrow) in a high-dimensional feature space, and in that space geometric relationships encode real ones: similar entities are nearby points, meaningful patterns are directions. So ML is geometry on data — classification draws boundaries between regions, clustering finds groups of nearby points, learning reshapes the space so data becomes separable — which matters because geometry is picturable, so you can reason about and debug models rather than just running them. The dot product is the universal similarity operation: it returns one number for how much two vectors align (large = same direction, zero = perpendicular), and its normalized form, cosine similarity, is the standard measure behind semantic search and recommendation (find vectors that point the same way). It's also the core of every model: a linear model's prediction is a dot product of learned weights with input features, and a neural-network layer is a matrix of dot products — which is why GPUs (machines for massively parallel dot products / matrix multiplies) power deep learning. A matrix is dual — both a transformation that reshapes space (how networks bend data to make it separable) and a container of data (a dataset is a matrix of row vectors), and because they're the same kind of object you can multiply them freely. Norms (L2/Euclidean, L1/Manhattan, Frobenius) measure a vector's size and the distance between data points — the "how big / how far" questions all over ML (regularization, loss, nearest-neighbors). Bonus: the whole quartet — represent as vectors, similarity via dot product, distance via norm, reshape via matrix — is the geometric foundation everything in ML builds on.

Where this connects #

Backward: This is the graphics course's 3D math (12.3) pointed at data instead of rendering — the same vectors, dot products, and matrices-as-transformations you learned to picture there, now the language of ML (the linear algebra that rotates a teapot classifies an image). The dot product returns from graphics (lighting) as the ML similarity operation. And "data is a matrix of vectors" is the 2D array you've handled as a programmer forever, now with geometric meaning.

Forward: This lesson's reframe (data is geometry) underlies the entire course and Year 4. The next lesson formalizes the space itself (vector spaces, independence, span, basis, dimension — the structure of the feature space). Later come the transformations that solve and decompose (systems, eigenvalues, and — the payoff — SVD and PCA, which find the meaningful directions in data's geometry). And the whole thing is the foundation of the ML to come: linear/logistic regression are dot products, neural networks are matrix transformations, and embeddings are vectors whose dot products encode meaning — everything you're about to learn is geometry in the space this lesson taught you to see.