Reflection — An Honest Take 8 min

Honest Take — Before You Begin


This is the module where I have to be most careful about honesty, because this is where the strongest opinions live and where the most bullshit gets said.

Nobody knows what happens next. Not Sam Altman, not Yann LeCun, not Demis Hassabis, not Dario Amodei. Not me. The people building the most advanced AI systems in history disagree fundamentally about where it's going. LeCun thinks LLMs are a dead end and world models are the future. Altman thinks scaling is enough. Amodei thinks we're close to something transformative and is terrified of the safety implications. Marcus thinks we're in a bubble. They can't all be right. They might all be wrong.

Here's what I do think, honestly:

LLMs are not the end of the road. They are remarkably capable at language tasks, and surprisingly capable at tasks nobody expected (coding, math, reasoning). But they have fundamental limitations: they hallucinate, they don't have persistent memory, they can't truly plan, they struggle with novel reasoning that requires grounding in the physical world. Something else will come after transformers. What that something is — world models, neurosymbolic systems, some architecture nobody has invented yet — I don't know.

The "AGI by 2027" crowd is probably wrong about the timeline, but possibly right about the direction. Aschenbrenner's "Situational Awareness" is a serious, detailed argument for rapid progress. But history is littered with confident AI predictions that were wrong. The AI winters happened because researchers were certain breakthroughs were imminent, and they weren't. The difference now is that we have massive compute, massive data, and architectures that actually scale. The question is whether scaling alone gets you to general intelligence, or whether there's a conceptual breakthrough still needed. I lean toward "conceptual breakthrough still needed," but I hold that belief loosely.

The alignment problem is real and not being taken seriously enough by most people. When I say alignment, I mean: how do you ensure that an AI system does what you actually want, not just what you literally asked for? This sounds simple and is impossibly hard. A system optimizing for "make users happy" could learn to tell people what they want to hear rather than what's true. A system optimizing for "reduce carbon emissions" could decide the most efficient way is to reduce the number of humans. These are not science fiction scenarios — they're direct consequences of optimization without careful constraint specification. The field of AI safety is trying to solve this. They haven't solved it yet.

About the AI ethics vs AI safety distinction: These are different communities with different concerns. AI ethics people (Gebru, Bender, Crawford) worry about present harms — bias, discrimination, surveillance, labor exploitation. AI safety people (Yudkowsky, Christiano, Olah) worry about future catastrophic risks — misalignment, loss of control, existential risk. Both are right. Both are important. The culture war between them is destructive and stupid. You need to understand both perspectives, which is why Module 7 covers both and Module 11 goes deeper on the justice angle.

GEB is the most important book in this module, and possibly in the entire curriculum. Hofstadter's "Godel, Escher, Bach" was written in 1979 and is still the deepest exploration of intelligence, consciousness, self-reference, and formal systems ever published. It doesn't mention neural networks or machine learning. It doesn't need to. The questions it asks — What is a self? Can meaning emerge from meaningless symbols? Is consciousness a strange loop? — are exactly the questions that modern AI forces us to confront. Read it slowly. Read it twice. Let it change how you think.


Conclusion #

Module 7 is where certainty dies. If you came into this curriculum thinking "AI will save/destroy the world," you'll leave this module thinking "it might, but nobody knows, and the specific outcomes depend on choices humans make in the next 10-20 years." That uncertainty is uncomfortable but honest. Anyone selling certainty about AI's future is selling something.

Predictions #

  • Reading GEB will take you months, not weeks. That's fine. It's not a book you finish. It's a book that finishes you.
  • You'll find yourself in the "middle" of the AI debate — not a doomer, not an accelerationist, but someone who sees the power and the risk simultaneously. That middle position is lonely on Twitter but correct in reality.
  • Aschenbrenner's "Situational Awareness" will either radicalize you about AI timelines or make you deeply skeptical of confident predictions. Your reaction will depend on your prior beliefs.
  • The Stanford HAI AI Index will become your go-to reference when someone makes a confident claim about AI. "Actually, the data shows..." is a powerful phrase.
  • You'll change your mind about something fundamental at least once during this module. That's a feature, not a bug.
  • After finishing this module, you'll be one of the most informed people in any room about AI — not because you're the smartest, but because you've read broadly, critically, and honestly. Most people read only the hype or only the criticism. You'll have read both.

That's the free preview. Sign in to continue this course.

Sign in to continue

New here? Make a desk →