R3 · dive 3
The road ahead
The reliable queue is the spine. Each later chapter adds one capability on top of it — and on the far side waits the system that applies them all.
R3 builds a queue that survives a crash. R4 through R8 each add one capability to that queue: a clock, a memory, restraint, frugality, endurance — and each is one more Redis structure used one more way. Then the patterns hand off to the system that puts them to work: the living EchoMQ course. Walk the arc on the right, then open the door. The engine the patterns sit below is /elixir.
The course arc · the reliable queue is the spine, each chapter adds one capability
The arc — one queue, one sense at a time
R3 builds a queue that is correct under a crash. A job moves from emq:{queue}:wait
into emq:{queue}:active and on to completion, and a worker that dies leaves its job
recoverable rather than lost. That correct queue is the floor. Every chapter after it adds exactly one
capability on top, and each capability is one more Redis structure put to one more use.
R4 · Time, Delay & Priority turns the sorted set into a clock. Delayed jobs are scored by their fire-time in a ZSET and swept by score; priority packs a tier and an arrival counter into one composite score. The queue learns when and in what order.
R5 · Streams & Events adds the durable log. The
emq:{queue}:events STREAM records every lifecycle transition, consumer groups read
it without losing their place, and event sourcing rebuilds state from the log. The queue learns to
remember and replay.
R6 · Flow Control & Scale keeps the tier stable under load. Rate limiters, worker concurrency, fair groups across tenants, and bulk batches stop demand from tipping the system over. The queue learns restraint.
R7 · Data Modeling & Memory attends to how the data lives in RAM. Compact encodings, short field names, capped structures, and the probabilistic structures that trade accuracy for memory. The queue learns to spend memory well.
R8 · Production & Operations runs the tier at scale: persistence, pooling, failover, and the Pinterest, Twitter, and Uber case studies, closing on the capstone door. The queue learns to survive production.
- R4 — the clock
- The ZSET scored by fire-time and packed priority — delay and order.
- R5 — the memory
- The events STREAM, consumer groups, and event sourcing — remember and replay.
- R6 — the restraint
- Rate limiters, concurrency, fair groups, and batches — stability under load.
- R7 — the frugality
- Encodings, short fields, capped and probabilistic structures — memory well spent.
- R8 — the endurance
- Persistence, failover, and the real case studies — surviving production.
Read the arc as one queue gaining one sense at a time: a clock, a memory, restraint, frugality, endurance. The reliable queue is the constant; each chapter is a verb it learns.
The door beyond — the living EchoMQ course
Having learned the patterns, the reader is ready for the system that applies them. The dedicated
EchoMQ course at /echomq teaches the polyglot job-queue protocol in
depth. It is a living, agile-spec course, built in two movements. Movement I (E1–E2) teaches
the as-built core library: the immutable wire protocol and data layer, then the lifecycle, components, and
three runtimes. Movement II (E3–E8) is a living spine that tracks the EMQ extension ladder —
emq.1 through emq.6 — rung by rung.
Each Movement-II page stands on the triangle: the redis-patterns pattern, the
emq.N implementation spec, and the as-built code. The page teaches from the spec
while the rung is drafted, and is re-grounded in the code when the rung ships. Every
→ EchoMQ door in this course opens onto a precise EchoMQ chapter:
# the redis-patterns door map (docs/echomq/echomq.toc.md) R0.3 → E1 # the protocol & the data layer R2 → E1 / E3 # the data layer & the extension substrate R3 → E2 / E5 / E6 # the lifecycle, batches, lifecycle controls R4 → E6 / E7 # lifecycle controls & EchoCache R5 → E6 / E7 # lifecycle controls & EchoCache R6 → E4 / E5 # groups & batches R8 → E8 # conformance, telemetry & the benchmark gate
R3 — this chapter — opens onto three EchoMQ chapters: E2, the lifecycle, components, and runtimes (the engine that runs the reliable queue); E5, batches (bulk consumption); and E6, lifecycle controls (TTL, cancel, and checkpoints). Pick a door on the right to read what waits behind it.
The redis-patterns → EchoMQ door map · pick a chapter that carries a door
emq.1–emq.6
specs exist). The served /echomq pages ship as the build lands. It is the living
companion course, not a course produced all at once.A door, not a depth
This is where the patterns hand off. Redis Patterns Applied has shown the techniques — the data structures, the atomic Lua, the four-layer model, the reliable queue — and grounded each in how EchoMQ-in-Portal really uses it. What it has deliberately not done is teach the polyglot protocol in full: the complete Lua inventory script by script, the three runtime implementations of the executor and the API, and the governance that keeps the wire frozen. That depth is a course of its own.
R0–R8 teach the Redis techniques applied. Each pattern lands twice — the technique and its trade-offs, then the one real EchoMQ excerpt that proves it. A catalog of judgement, grounded.
The living EchoMQ course teaches the whole protocol in depth and tracks the EMQ build — E1–E8
standing on emq.1–emq.6, the spec, and the as-built
code together.
The patterns are the vocabulary; EchoMQ is the language. Learn the patterns here, then go build with them there.
The forward door
Start building with the EchoMQ course
The polyglot protocol taught end to end and tracked as it is built. R3 opens onto three of its chapters: the engine that runs the reliable queue, the batch path that consumes in bulk, and the lifecycle controls that cancel, time out, and checkpoint a running job. Begin at the course home, or step straight to R3's doors.
Return to the chapter R3 · Reliable Queues to review the reliable-queue family in one place, or carry on into the pattern chapters that follow.
References
Where the structures the later chapters teach are specified, the protocol the whole arc builds on, the machine-readable map both courses publish, and the dives this orientation closes.
Sources
- Redis — Streams — the durable, replayable log R5 teaches on top of the reliable queue:
XADD, consumer groups,MAXLEN ~trimming. - Redis — Sorted sets — the ZSET as a clock and a priority ladder, the structure R4 turns into delay and composite priority.
- BullMQ — the queue protocol the whole arc builds on and that EchoMQ ports across three runtimes.
- llms.txt — The /llms.txt convention — the machine-readable map format both this course and the EchoMQ course publish for agents.
Related in this course
- R3 · Reliable Queues — the chapter this dive closes.
- R3 · States as locations — dive 2: the lifecycle as one atomic Lua move, the foundation the later chapters build on.
- R0.3 · The door to EchoMQ — the course's first EchoMQ door: the cross-runtime contract.
- R0.2 · Redis under Portal — where Redis sits in Portal and the reserved multi-runtime tier the queue patterns fill.