The Queue · module 02
Jobs, lanes & the consumer
The lifecycle is the machine. This module is the people who run it — the producer, the fair worker, and the loop that owns the rhythm.
A job enters the machine through one verb and leaves through another, but between them stand three roles. The producer admits work with one idempotent script. The fair worker claims in turn off a constructed ring, so no identity starves the rest. The loop owns the cadence — a supervised process that parks rather than polls, costing the wire nothing while it waits. Pick a role below to read the real surface it owns, then take the three dives.
Three roles over one wire
A job's working life runs through three hands: the producer that admits it, the worker that claims it fairly,
and the loop that decides when to look. Each is a real surface in echo/apps/echo_mq.
Pick a role to read the verb it owns, what it touches, and the verdict it returns.
The three roles · pick one
Three roles, three verbs, one wire: admit once, claim in turn, and look only when there is reason to.
The dives
Three dives read the roles in depth, each grounded in real echo/apps/echo_mq code:
the idempotent enqueue and the flat claim, the rotating ring that builds fairness, and the supervised loop that
parks on the wake key.
Enqueue & claim
The @enqueue two-beat — the EMQKIND kind-gate, EXISTS idempotency, the three-field row, the pending entry — and the flat @claim that pops, leases, and returns.
Fair lanes & the ring
Per-group lanes and a rotating ring. @gclaim does LMOVE LEFT RIGHT to rotate one step, then serves that lane's head — fairness constructed, not hashed.
The consumer loop
The supervised reap → promote → drain → park beat. BLPOP the wake key — park, don't poll — and a raising handler becomes a typed retry, not a crash.
Pattern & implementation
Where Redis Patterns Applied teaches the move — a reliable queue that admits work once and hands it to exactly one consumer — this module is where the move lands, with fairness and a parking loop built on top.
A reliable queue admits each item once and delivers it to exactly one worker, which claims, runs, and acknowledges — blocking for the next item rather than busy-polling. Reliable queues teaches it.
EchoMQ.Jobs.enqueue/4 admits idempotently; EchoMQ.Lanes.claim/3
serves identities in turn off a rotating ring; EchoMQ.Consumer runs the loop and
parks on the wake key with BLPOP.
Take the dives in order, or jump to the consumer loop if you already know enqueue and claim. The next module — Batches — reads how the same per-item contract is admitted and extended in bulk, in one wire flush.
References
Sources
- Valkey — LMOVE — the atomic rotate the ring is turned with, one step per claim.
- Valkey — ZPOPMIN — popping the oldest pending id the claim leases.
- Valkey — BLPOP — the blocking pop the consumer parks on instead of polling.
- Redis — EVALSHA — load-once, run-by-SHA dispatch of each declared-keys script.
Related in this course
- The Queue — the pillar this module belongs to.
- Enqueue & claim — the producer and the flat worker.
- The Lua layer — the scripts the verbs run.
- redis-patterns · Reliable queues — the reliable-queue / consumer pattern.