Streaming, Data Infrastructure & Advanced Topics covers: Streaming Systems, Designing Distributed Systems, Rethinking IoT. Understand event streaming, message queues, data pipelines, and advanced infrastructure patterns. Connect the dots between all previous modules.
Rails is traditionally request-response, but modern Rails apps increasingly use event-driven patterns. ActiveJob + Sidekiq is a simple job queue. Kafka/SQS is for when you outgrow Sidekiq or need cross-service communication.
| Rails Pattern | Streaming Equivalent |
|--------------|---------------------|
| ActiveJob.perform_later | Kafka producer: event.publish |
| Sidekiq worker | Kafka consumer: event.consume |
| after_commit callback | Event emission on state change |
| Webhooks | Kafka topic subscription |
| Polling for changes | Change Data Capture (CDC) |
This course unlocks once you've finished its prerequisite. Open prerequisite →
Kafka and event streaming feel distant from Rails. Rails is request-response. A user clicks a button, a controller runs, a view renders, done. Kafka is a distributed commit log pr…
Here's a question your Sidekiq dashboard can't answer: an order was placed at 14:03, but the worker processed it at 14:09 because a deploy restarted the queue. If you're counting …
In Course 4 you read the first half of this book and learned the patterns that shape a single request's path — sidecars, ambassadors, load-balanced replicas. All of those patterns…
Every architecture idea you've met in this course — durable logs, work queues, dead-letter handling — assumed a comfortable fact: your producers are a handful of well-behaved Rail…
Work through each item before the checkpoint.
Build an event-driven system on a real Kafka broker: a producer publishes order events, a consumer processes them into analytics and notifications, poison messages land in a dead-…
6 lessons. Read in order; spiral back when you need to. By the end you'll have used the core ideas twice — once on the abstract, once on something you'll meet at work next week.