How Software Gets Built: Methodologies, Version Control, and CI/CD

Hook #

The previous course made your code correct. This course is about making the process that produces the code an engineering discipline — because at any real scale, the biggest determinant of software quality is not how good any individual is, but how the team's system of work is designed. This first lesson maps that system: the methodologies that structure how work is planned and sequenced (waterfall vs. agile vs. scrum vs. kanban), the version control workflows that let many people change the same codebase without chaos, and the CI/CD pipeline that carries a change from a developer's laptop to production automatically and safely. The through-line — and the single most counterintuitive finding in modern software engineering — is that smaller, more frequent changes are safer than large, careful ones. That claim reverses most people's intuition (surely big careful releases are safer?), and it is the empirically-backed foundation under everything from git branching strategy to why your team deploys ten times a day instead of once a quarter.

What you'll be able to do by the end of this lesson #

• Distinguish the major development methodologies — waterfall's up-front sequencing, agile's iterative feedback, scrum's time-boxed sprints, kanban's continuous flow — and explain what problem each was designed to solve.
• Explain why small, frequent, integrated changes reduce risk more than large, infrequent ones, and connect it to the DORA metrics that measure it.
• Describe a CI/CD pipeline end to end (commit → automated build → test → deploy) and why automating the path to production is a safety mechanism, not just a convenience.
• Reason about version-control workflows (trunk-based vs. long-lived feature branches) in terms of integration frequency and merge risk.

A quick try before we start #

Two teams ship the same feature. Team A works on a branch for three weeks, then merges a 4,000-line diff. Team B merges ten times over those three weeks, ~400 lines each, behind a feature flag. Which team has the riskier final merge, and why? Team A — by a wide margin. Their branch has drifted three weeks away from everyone else's work, so the final merge is a giant, conflict-ridden, hard-to-review event where bugs hide in the noise, and if something breaks, the blast radius is 4,000 lines to bisect. Team B integrated continuously, so each merge is small, reviewable, and independently revertible, and the codebase never diverged far from reality. This is the whole argument for continuous integration in one comparison: the pain of integration grows superlinearly with the size and age of the change, so the answer is to integrate constantly and keep every change small. Notice this is a statement about risk management, not about tooling.

Why this matters here #

This lesson opens Software Engineering Practices by establishing the system that the rest of the course's practices live inside. Design patterns, SOLID, clean architecture, DDD (the later lessons) are all techniques for structuring code — but they only pay off inside a process that integrates, tests, and ships that code safely and frequently. A beautifully architected system that takes three months to release is still a slow, risky system. This lesson also directly reinforces and extends earlier material: git internals (the objects-and-DAG model from the Missing Semester course) becomes the collaboration workflow here, and the systematic-debugging discipline from Course 11.1 (git bisect, regression tests) is far more powerful when history is a stream of small, well-integrated commits than a few giant merges.

For a working engineer, understanding the process system is what separates someone who participates in a team's workflow from someone who can improve it. Every team has a methodology (even if it's "chaos"), a branching strategy, and a deploy process — and these choices determine how fast the team can move and how often it breaks things. The engineer who understands why trunk-based development and continuous integration reduce risk, and can point to the DORA evidence, is the one who can advocate for changes that make the whole team faster and safer. And the counterintuitive core — small and frequent beats large and careful — is one of those ideas that, once internalized, changes how you work at every level, from how you size a PR to how you argue for a deploy pipeline.

The engineer's lens #

The central and most counterintuitive idea is that batch size is the master variable: small, frequent changes are safer than large, infrequent ones, and almost every good practice in this lesson is a consequence of that single fact. The intuition most people carry — "big releases are risky, so be extra careful and thorough with them" — has the causality backwards. The risk isn't inherent to releasing; it's created by the batch size. A large change is risky because it's large: more code to review (so review quality drops), more that can interact and conflict, a bigger blast radius when it breaks, and a harder debugging job (more places the bug could be) when it does. Shrink the batch and every one of those risks shrinks with it — a 50-line change is trivially reviewable, its blast radius is 50 lines, and if it breaks you know exactly where to look. This is why the DORA research (from Accelerate) found that elite teams deploy more often, not less, and have lower failure rates — the two go together because small batches are the common cause of both. It reframes continuous integration, trunk-based development, and CI/CD not as fashionable tooling but as different expressions of one risk-management principle: keep the change small, integrate it constantly, and ship it fast so feedback arrives while the context is still fresh. Once you see batch size as the master variable, you evaluate every process question — "should this be one PR or five?", "should we branch for a week or merge daily?" — through it.

The second lens is the methodologies as answers to a single question — how do you handle the fact that you don't know everything up front? — arranged on a spectrum from "pretend you do" to "assume you don't." Waterfall (design fully, then build, then test, then ship) is the "pretend you know" end: it works when requirements are genuinely fixed and well understood (some hardware, some regulated domains) and fails catastrophically when they aren't, because you discover the requirements were wrong only at the end, when change is most expensive. Agile is the philosophical answer to waterfall's failure: assume you'll learn as you go, so work in short iterations that each produce something real, get feedback, and adjust — trading the illusion of a predictable plan for the reality of fast course-correction. Scrum and kanban are two concrete implementations of the agile idea: scrum time-boxes work into fixed-length sprints with ceremonies (planning, standup, review, retro) and is good at rhythm and commitment; kanban drops the time-box for continuous flow with work-in-progress limits and is good at steady streams of varied work (support, ops, maintenance). The engineer's takeaway isn't "agile good, waterfall bad" — it's understanding what problem each solves so you can recognize when a team is using the wrong one (a research-heavy project forced into two-week sprints, or an unpredictable ops load forced into rigid planning). The methodology is a tool for managing uncertainty, and the right tool depends on how much uncertainty you actually have.

The third lens is CI/CD as the automation of trust — the pipeline exists so that "it works" is proven mechanically rather than asserted hopefully. Continuous Integration means every change is automatically built and tested the moment it's pushed, so integration problems surface in minutes rather than at a painful merge weeks later — it's the machine enforcing the small-batch discipline. Continuous Delivery/Deployment extends the automation all the way to production: the same pipeline that tested the change can ship it, so releasing stops being a fraught manual event (with a runbook, a war room, and prayer) and becomes a routine, low-drama, easily-reverted push. The deep point is that automating the path to production is a safety mechanism, not a convenience: a manual deploy is error-prone (someone forgets a step at 2 AM), unrepeatable, and scary — and scary things get done rarely, which makes batches large, which brings back all the risk. An automated pipeline is repeatable, fast, and boring, and boring is safe — it lets you deploy small changes constantly, which is the whole game. For a Rails engineer this is concrete and familiar: the GitHub Actions workflow that runs your RSpec suite and Brakeman on every push, and the deploy tooling (Kamal, Capistrano, a PaaS) that ships on merge, together are the CI/CD pipeline — and every flaky test or manual deploy step is friction that pushes the team back toward large, risky batches. Seeing the pipeline as the mechanism that makes small-batch delivery possible is the connection that ties this whole lesson together.

What to focus on in the resources #

• The DevOps Handbook, Parts III-IV — primary. Read CI/CD as a discipline built on flow, feedback, and small batches — not as a list of tools. The "three ways" framing is the transferable mental model.
• Accelerate's DORA metrics. Learn the four (deploy frequency, lead time for changes, change-failure rate, time-to-restore) and, crucially, why they move together — this is the evidence for the small-batch thesis. It's the data that lets you argue for process change.
• Pro Git's workflow chapters. Focus on trunk-based vs. long-lived branches through the lens of integration frequency. You already know git's internals; here it's the collaboration strategy that matters.
• Skip on first pass: specific CI-tool YAML syntax, scrum-certification ceremony minutiae, and the agile-manifesto history. Get the batch-size principle, the methodology spectrum, and the CI/CD-as-safety idea.

Explain it back #

Explain to a colleague why a team that deploys ten times a day can be safer than one that deploys once a quarter. A strong answer: batch size is the master variable. A large, infrequent release is risky because it's large — more code to review (so review gets worse), more interactions and conflicts, a bigger blast radius, and a harder debugging job when it breaks. Small, frequent changes shrink every one of those risks, which is why the DORA research found elite teams deploy more often and fail less — the two come from the same cause. Continuous integration (auto-build-and-test every push) enforces small batches by surfacing integration problems in minutes; CI/CD automates the path to production so deploys become routine and revertible instead of scary manual events — and scary events happen rarely, which forces batches large again. Bonus: place the methodologies on the uncertainty spectrum (waterfall assumes you know the requirements up front; agile/scrum/kanban assume you'll learn as you go) and say what problem each solves.

Where this connects #

Backward: The version-control workflow here is the collaboration layer on top of git's objects-and-DAG internals from the Missing Semester course. The whole lesson extends Course 11.1's individual practices to the team: git bisect and regression tests (11.1's debugging lesson) are far more powerful over a stream of small commits, and code review (11.1's correctness lesson) becomes a pipeline gate here.

Forward: The next lesson (the testing pyramid, TDD, BDD) is what fills the CI pipeline — the automated tests that make continuous integration trustworthy. The design lessons that follow (patterns, SOLID, clean architecture, DDD) structure the code that flows through this pipeline. And technical debt and metrics (the final lesson) are how you keep the system healthy as it grows, so the pipeline stays fast and the batches stay small.