Parallel and Concurrent Algorithms covers: Concurrency vs Parallelism, The Work-Span Model, Parallel Patterns, Concurrency Models, Lock-Free, Wait-Free, and the GPU Frontier. Year 1, Quarter 4. Includes 11 exercises and 1 projects.
This course unlocks once you've finished its prerequisite. Open prerequisite →
Here's a question most software engineers fail despite having years of experience:
You've parallelised your code across 8 cores. The serial version took 100 seconds; the parallel version takes... 35. Not 12.5 (100/8). Not even 20. Thirty-five seconds. Where did …
The previous lesson said parallelism is bounded by an algorithm's span — the longest chain of dependent operations. The natural follow-up: what algorithms have small span? A handf…
Three big ideas have emerged for organising concurrent programs:
Locks are slow when contended. A mutex held for 10 nanoseconds in the uncontested case can stall a thread for 100 microseconds when 32 cores all want it. Lock-free data structures…
- [ ] Implement parallel map using Ruby Threads — divide array into chunks, process in threads - [ ] Implement parallel sum using Elixir Task.async — spawn tasks for array chunks,…
- [ ] Implement parallel merge sort in Ruby using fork/Process — Split array, sort in child processes - [ ] Build a thread pool in Ruby — fixed number of worker threads processing…
- [ ] Implement a MapReduce framework in Ruby — map phase distributes to workers, shuffle, reduce aggregates - [ ] Build a concurrent web crawler in Elixir — GenServer pool, rate …
- [ ] Build a parallel pipeline in Elixir — Stage 1: read file chunks (GenStage producer), Stage 2: parse/transform (producer-consumer), Stage 3: aggregate (consumer); compare seq…
Implement a mini MapReduce framework in Ruby. Map phase: split a large text file into chunks, spawn worker threads that emit (word, 1) pairs. Shuffle phase: group by key. Reduce p…
- [ ] What is the difference between concurrency and parallelism? - [ ] Explain the three concurrency models: shared memory (threads/locks), message passing (actors), and CSP (cha…
11 lessons. Read in order; spiral back when you need to. By the end you'll have used the core ideas twice — once on the abstract, once on something you'll meet at work next week.