The Queue · module
Lifecycle controls
Enqueue, claim, complete is the spine. The controls are everything an operator and a worker reach for around it.
Beyond the state machine sit the controls over its edges: over time — schedule a
job for later, retry it on a curve, run it on a cadence; over the worker in hand — cancel a long
handler cooperatively, checkpoint its lease so it is not reaped mid-work, recover the ones that stalled; and over
the whole queue — pause claiming, drain the backlog, obliterate a dead queue, and reach into one
job's row to fix it. Each is a real transition in echo/apps/echo_mq — a score, a pure
function, a fenced script. Pick a control below, then take the dives.
Three scopes of control
The controls divide into three scopes — time, the worker in hand, and the whole queue. Pick a control to read the real module function it names, the structure it touches, and the one-line effect. The map is the module.
The control map · pick a control
Every control is a real function over the owned keyspace — a score moved, a curve computed, a field set. Learn the map and you have read the control plane.
The dives
Three dives read the control plane in depth, each grounded in real echo/apps/echo_mq
code: control over time, over the worker in hand, and over the whole queue.
Scheduling & recurrence
enqueue_at / enqueue_in park a job on the schedule set at a run-at score; promote releases the due ones; Backoff is the pure curve; Repeat is the cadence.
Cancellation & checkpoints
The cooperative cancellation token; extend_lock checkpoints the lease so a long handler is not reaped; the count-thresholded stalled sweep recovers or dead-letters.
The operator plane
Admin pause / resume / drain / obliterate over the whole queue, and the per-job verbs on Jobs — update the payload, write progress, append logs, remove, reprocess.
Pattern & implementation
Where Redis Patterns Applied teaches the delay, schedule, and priority family — a ZSET scored by run-at time, a promote pass, a backoff curve — this module is where that family lands as a control plane you own.
Delay, schedule, and priority over Redis: a sorted set scored by run-at time, swept by a promote pass, with a retry curve computed client-side. Time, delay & priority teaches the family.
The same machine, owned: @schedule parks on the schedule set,
@promote releases the due ones on the server clock, Backoff is
the curve, and EchoMQ.Admin is the operator plane above it.
Take the dives in order, or jump to the operator plane if you are here for the runbook. The control plane sits on the protocol — the keyspace and the Lua layer every control runs on.
References
Sources
- Valkey — ZADD — the schedule-set insertion at the run-at score.
- Valkey — ZRANGEBYSCORE — the due read the promote pass runs.
- Redis — EVALSHA — the load-once dispatch every control verb runs by SHA.
- Valkey — Documentation — the substrate of record EchoMQ is backed by.
Related in this course
- The Queue — the chapter this module belongs to.
- The Protocol — the keyspace and the Lua layer the controls run on.
- The Lua layer — how a control transition is one atomic script.
- redis-patterns · Time, delay & priority — the delay/schedule/priority family.
- redis-patterns · Reliable queues — the lease/recover machinery the controls extend.