jonnifyredis

R3 · dive 3

The road ahead

The reliable queue is the spine. Each later chapter adds one capability on top of it — and on the far side waits the system that applies them all.

R3 builds a queue that survives a crash. R4 through R8 each add one capability to that queue: a clock, a memory, restraint, frugality, endurance — and each is one more Redis structure used one more way. Then the patterns hand off to the system that puts them to work: the living EchoMQ course. Walk the arc on the right, then open the door. The engine the patterns sit below is /elixir.

The course arc · the reliable queue is the spine, each chapter adds one capability

R3 the spine reliable queue R4 · Time, Delay & Priority R5 · Streams & Events R6 · Flow Control & Scale R7 · Data Modeling & Memory R8 · Production & Operations

The arc — one queue, one sense at a time

R3 builds a queue that is correct under a crash. A job moves from emq:{queue}:wait into emq:{queue}:active and on to completion, and a worker that dies leaves its job recoverable rather than lost. That correct queue is the floor. Every chapter after it adds exactly one capability on top, and each capability is one more Redis structure put to one more use.

R4 · Time, Delay & Priority turns the sorted set into a clock. Delayed jobs are scored by their fire-time in a ZSET and swept by score; priority packs a tier and an arrival counter into one composite score. The queue learns when and in what order.

R5 · Streams & Events adds the durable log. The emq:{queue}:events STREAM records every lifecycle transition, consumer groups read it without losing their place, and event sourcing rebuilds state from the log. The queue learns to remember and replay.

R6 · Flow Control & Scale keeps the tier stable under load. Rate limiters, worker concurrency, fair groups across tenants, and bulk batches stop demand from tipping the system over. The queue learns restraint.

R7 · Data Modeling & Memory attends to how the data lives in RAM. Compact encodings, short field names, capped structures, and the probabilistic structures that trade accuracy for memory. The queue learns to spend memory well.

R8 · Production & Operations runs the tier at scale: persistence, pooling, failover, and the Pinterest, Twitter, and Uber case studies, closing on the capstone door. The queue learns to survive production.

R4 — the clock
The ZSET scored by fire-time and packed priority — delay and order.
R5 — the memory
The events STREAM, consumer groups, and event sourcing — remember and replay.
R6 — the restraint
Rate limiters, concurrency, fair groups, and batches — stability under load.
R7 — the frugality
Encodings, short fields, capped and probabilistic structures — memory well spent.
R8 — the endurance
Persistence, failover, and the real case studies — surviving production.

Read the arc as one queue gaining one sense at a time: a clock, a memory, restraint, frugality, endurance. The reliable queue is the constant; each chapter is a verb it learns.

The door beyond — the living EchoMQ course

Having learned the patterns, the reader is ready for the system that applies them. The dedicated EchoMQ course at /echomq teaches the polyglot job-queue protocol in depth. It is a living, agile-spec course, built in two movements. Movement I (E1–E2) teaches the as-built core library: the immutable wire protocol and data layer, then the lifecycle, components, and three runtimes. Movement II (E3–E8) is a living spine that tracks the EMQ extension ladder — emq.1 through emq.6 — rung by rung.

Each Movement-II page stands on the triangle: the redis-patterns pattern, the emq.N implementation spec, and the as-built code. The page teaches from the spec while the rung is drafted, and is re-grounded in the code when the rung ships. Every → EchoMQ door in this course opens onto a precise EchoMQ chapter:

# the redis-patterns door map (docs/echomq/echomq.toc.md)
  R0.3 → E1          # the protocol & the data layer
  R2   → E1 / E3     # the data layer & the extension substrate
  R3   → E2 / E5 / E6 # the lifecycle, batches, lifecycle controls
  R4   → E6 / E7     # lifecycle controls & EchoCache
  R5   → E6 / E7     # lifecycle controls & EchoCache
  R6   → E4 / E5     # groups & batches
  R8   → E8          # conformance, telemetry & the benchmark gate

R3 — this chapter — opens onto three EchoMQ chapters: E2, the lifecycle, components, and runtimes (the engine that runs the reliable queue); E5, batches (bulk consumption); and E6, lifecycle controls (TTL, cancel, and checkpoints). Pick a door on the right to read what waits behind it.

The redis-patterns → EchoMQ door map · pick a chapter that carries a door

A note on honesty. The EchoMQ course is designed — its spec system and TOC are authored — and the EMQ ladder is drafted (the emq.1–emq.6 specs exist). The served /echomq pages ship as the build lands. It is the living companion course, not a course produced all at once.

A door, not a depth

This is where the patterns hand off. Redis Patterns Applied has shown the techniques — the data structures, the atomic Lua, the four-layer model, the reliable queue — and grounded each in how EchoMQ-in-Portal really uses it. What it has deliberately not done is teach the polyglot protocol in full: the complete Lua inventory script by script, the three runtime implementations of the executor and the API, and the governance that keeps the wire frozen. That depth is a course of its own.

the patterns

R0–R8 teach the Redis techniques applied. Each pattern lands twice — the technique and its trade-offs, then the one real EchoMQ excerpt that proves it. A catalog of judgement, grounded.

the system

The living EchoMQ course teaches the whole protocol in depth and tracks the EMQ build — E1–E8 standing on emq.1–emq.6, the spec, and the as-built code together.

The patterns are the vocabulary; EchoMQ is the language. Learn the patterns here, then go build with them there.

The forward door

Start building with the EchoMQ course

The polyglot protocol taught end to end and tracked as it is built. R3 opens onto three of its chapters: the engine that runs the reliable queue, the batch path that consumes in bulk, and the lifecycle controls that cancel, time out, and checkpoint a running job. Begin at the course home, or step straight to R3's doors.

Return to the chapter R3 · Reliable Queues to review the reliable-queue family in one place, or carry on into the pattern chapters that follow.

References

Where the structures the later chapters teach are specified, the protocol the whole arc builds on, the machine-readable map both courses publish, and the dives this orientation closes.

Sources

  • Redis — Streams — the durable, replayable log R5 teaches on top of the reliable queue: XADD, consumer groups, MAXLEN ~ trimming.
  • Redis — Sorted sets — the ZSET as a clock and a priority ladder, the structure R4 turns into delay and composite priority.
  • BullMQ — the queue protocol the whole arc builds on and that EchoMQ ports across three runtimes.
  • llms.txt — The /llms.txt convention — the machine-readable map format both this course and the EchoMQ course publish for agents.

Related in this course