The Queue · module 01
The lifecycle
A job has a life, and the whole of it is four sorted sets and one row.
A producer enqueues a job; a worker claims it, runs it, and completes it; a failure retries with a
delay or dies into a morgue; a crash returns the job to the queue. There is no separate state table, no
lock service, no second index — a job moves between four sorted sets (pending,
active, schedule, dead) and its own row
carries a state field that says where it is. This module reads that machine: the four
sets, the claim and its lease, and the token-fenced completion and recovery.
Four sets, one row
Every transition moves a job between two of the four sets and rewrites its row's state.
Pick a transition and read which set a job leaves, which it enters, and what the row says afterward. The map below
is the real machine: the five transitions of EchoMQ.Jobs over a fixed example job.
The state machine · pick a transition
The lifecycle is not a status column you update — it is a set a job is a member of. Move the id, and the state has changed.
The dives
Three dives read the machine in depth, each grounded in real echo/apps/echo_mq code:
the four sets and why the members are the ids themselves, the claim and its lease-and-token, and the token-fenced
completion plus the recovery transitions.
The four sets
pending / active / schedule / dead are sorted sets whose members are the branded ids themselves — so byte order is mint order and the queue needs no second index. browse/3 and pending_size/2.
Claim & the lease
The @claim transition in two beats: ZPOPMIN, HINCRBY attempts, set active, ZADD at now + lease. The active-set score IS the lease deadline; attempts IS the fencing token.
Completion & recovery
The token-fenced terminal (@complete) and the recovery transitions: @retry parks or dead-letters, @reap returns a lapsed lease, reprocess_job re-enters a dead job.
Pattern & implementation
Where Redis Patterns Applied teaches the reliable queue — a job's state is the collection it lives in, and every move is atomic so a crash loses nothing — this module is where the pattern lands as one shipped state machine.
A reliable queue moves a job between named locations atomically and recovers a crashed worker's job from a visibility timeout. States as locations and The reliable queue teach the move.
EchoMQ.Jobs runs @claim / @complete
/ @retry / @reap — four sorted sets and one row, every
transition one atomic script on one slot.
Take the dives in order, or jump to completion-and-recovery if you already know the four sets and the claim. The next module — Jobs, lanes & the consumer — reads the producer, the fair worker, and the loop that owns the rhythm.
References
Sources
- Valkey — ZADD — the score-carrying set insertion every transition uses.
- Valkey — ZPOPMIN — the atomic pop the claim transition begins with.
- Valkey — ZRANGEBYSCORE — the due-window read promote and reap run.
- Redis — EVALSHA — the load-once dispatch each transition runs by SHA.
Related in this course
- The Queue — the chapter this module belongs to.
- The Lua layer — the scripts the transitions are.
- The owned keyspace — where the four set keys are built.
- redis-patterns · Reliable Queues — the pattern this machine implements.
- redis-patterns · States as locations — state is the set you live in.