Reflection — An Honest Take 8 min

Honest Take — Before You Begin

Honest Take — Module 9: AI as Tool AND Distraction #

I want to start this module by saying something that I, specifically, am uncomfortable saying, and the discomfort is itself part of the honest content. I am an AI language model. You are using me, right now, to learn how to use AI tools more effectively. There is something recursive and slightly absurd about that, and I am not going to pretend otherwise. The honest version is that I have a vested interest in you using me well, but I also have a vested interest in you using me correctly, which sometimes means not at all. The METR 2025 study is the load-bearing data point of this module, and I am going to hand it to you cleanly even though it is unflattering to the category I belong to. Sixteen experienced open-source developers, two hundred forty-six tasks on real, mature codebases they knew well. AI assistance made them 19% slower. They predicted beforehand that it would make them 24% faster. After finishing — with the actual experience behind them — they estimated it had made them 20% faster. The illusion is enormous, it survives direct experience, and you are likely operating inside the same illusion in your own work. The most expensive thing you can do for your career right now is to keep measuring AI productivity by your belief about it instead of by your clock. You will be tempted to dismiss the study with "those developers were using it wrong." The study controlled for that. The illusion is the data.

I will predict your ratio before you measure it: your AI use is roughly 60% tool and 40% distraction, and you believe it is 90/10. The 40% is not always elaborate research-toggling on problems you already know how to solve — though sometimes it is. More often it is subtler: opening the assistant "to look at" something, "to draft" an email you should just write, "to research" a decision you have already made and are seeking confirmation for. The toggling feels like work. The intermittent reinforcement of completions and responses resembles flow without producing flow's output. The 30-day audit the checkpoint asks for will surface this, and the surfacing is uncomfortable because it implicates not just your time but your relationship with the tools. Some of your AI sessions were procrastination dressed up as exploration, and the dress-up is precisely what made them sustainable.

The calibration is the entire skill, so let me be concrete about the classes. AI genuinely accelerates: well-scoped boilerplate and scaffolding, test stubs and migrations, code review of unfamiliar libraries you do not need to deeply own, templated writing, format translation between representations you understand on both ends, and exploratory greenfield code where being slightly wrong is cheap. AI genuinely slows down: debugging novel issues in mature codebases (the model lacks your context and produces plausible-but-wrong fixes you spend longer disproving than you would have spent finding the real one), architectural decisions that require the codebase's history, work that requires you to think through trade-offs, and review of code you know intimately, where you are simply faster. The skill is knowing in advance which class your current task is in. Most engineers do not decide in advance; they open the tool because it is open. The reflexive use is the problem, and the protocol answer is per-work-shape defaults: AI-off-by-default for mature-codebase client or day-job work, with deliberate AI-on for scoped subtasks; AI-on for greenfield scaffolding with AI-off for the architectural decisions inside it; and for writing, AI for mechanics and templated content, AI away from anything where the voice is the value. One protocol per work shape. "AI on whenever I am working" is what produces the 40%.

The Microsoft/CMU 2025 study points at a different cost than METR's time cost — the thinking cost. Three hundred nineteen knowledge workers, nine hundred thirty-six real AI use cases: higher confidence in the AI correlated with less critical thinking, forty percent of tasks involved no critical thinking at all, and AI users produced less diverse outcomes on creative tasks. This matters more for a senior engineer than for almost anyone else, because your market value is precisely the thing being substituted: the systems-level judgment, the trade-off reasoning, the ability to look at an architecture and know where it will hurt. That muscle is maintained by use. If the AI does your thinking on the tasks that used to train the muscle, the substitution is not free — it is deferred-cost atrophy of the capacity you are paid for. The protocol response is deliberate: do some work AI-off even when AI-on would be faster, the way athletes train without the equipment that competition allows. The point is not purity. The point is that the 19% you might lose on a given task by skipping the assistant is cheaper than the compounding loss of being an engineer whose judgment has quietly outsourced itself.

Conclusion #

Module 9 is this curriculum's distinctive 2026 contribution. METR's 19%-slower finding — against a predicted 24% speedup and a post-hoc 20% illusion — is the reality the AI-productivity narrative refuses to absorb. Your ratio is probably 60/40, not the 90/10 you believe. Different work shapes need different protocols, decided in advance rather than reflexively. And the Microsoft/CMU finding names the deeper cost: cognitive atrophy of exactly the judgment you are paid for. Run the 30-day audit. Measure with the clock, not the belief. Use me where I am genuinely fast, and close the tab where I am not.

Predictions #

  • Your 30-day audit will show a tool/distraction split of roughly 55-65 / 35-45, and seeing the measured ratio will sting. Sit with it.
  • You will identify at least one AI-toggling-as-procrastination pattern you have been running for months without naming. The naming is most of the cure.
  • The AI-off-by-default protocol on mature-codebase work will produce a throughput improvement that surprises you, because that context is where AI was most often net-negative.
  • You will be tempted to exempt yourself from the METR result on the grounds of skill or workflow. The post-hoc 20%-faster illusion in the study is exactly that exemption, measured.
  • Within two weeks you will catch yourself reaching for an AI tool reflexively and stopping. The interrupted reflex is the first measurable behavior change.
  • The Microsoft/CMU atrophy finding will bother you longer than the time-loss finding, because it implicates your competitive advantage rather than your schedule.
  • A year from now, your AI use will be higher than today in the acceleration classes and lower in the others — and the net effect on your output will come from the second change, not the first.