The Queue · module 03
Batches
A batch is not a new kind of work. It is the same per-item contract — pipelined.
To admit many jobs is to run the @enqueue transition many times in
one wire flush; to compose many flows is to land each through the single add/3,
fail-closed per flow; to checkpoint many leases is to re-score every active member under
one server-clock read. Each bulk verb is the bulk twin of a single verb taught elsewhere, and
each answers one verdict per item, in input order. Batching changes the round-trip count, never
the rule each item obeys. Pick a verb below, then take the three dives.
Three bulk verbs, one rule
Three verbs on the real Queue surface admit, compose, and extend many at once. Each is the bulk twin of a single verb, each saves round trips, and each keeps the single verb's contract. Pick a verb to read its handle, the single verb it batches, its wire shape, and the shape it returns — the readout is a pure function of the verb chosen.
The bulk verbs · pick one
Each bulk verb is the bulk twin of a single verb: it saves round trips and keeps the contract. A batch is a wire optimization, not a semantic one.
The dives
Three dives read the bulk verbs in depth, each grounded in real echo/apps/echo_mq
code: admitting many in one flush, composing many flows fail-closed per flow, and extending many leases under
one clock read.
Enqueue many
enqueue_many/3 loads @enqueue once with SCRIPT LOAD, then pipelines one EVALSHA per item — per-item verdicts in input order, under the same gate, row, and idempotency as enqueue/4.
Bulk flows
Flows.add_bulk/3 lands many flows, each via the single add/3, fail-closed per flow — a flow that fails leaves its own parent held; the batch continues, returning the landed flows in input order.
Batch lease extension
extend_locks/4 re-scores every active member whose token matches under one server-clock read, returning the failed list — contrasted with the single extend_lock/5, which errors on a stale token.
Pattern & implementation
Where Redis Patterns Applied teaches pipelining — many commands in one round trip, replies in order — this module is where the move lands across three verbs.
Pipelining sends many commands in one round trip and reads the replies in order, so N operations cost one network turn instead of N. Reliable queues is the near side of the door.
EchoMQ.Jobs.enqueue_many/3 loads @enqueue once and
pipelines the EVALSHA calls; EchoMQ.Flows.add_bulk/3 issues
add/3 per flow; EchoMQ.Jobs.extend_locks/4 re-scores every
matching member in one script — each keeping the single verb's contract.
Take the dives in order, or jump to batch lease extension if you already know admission. The next module in The Queue covers the control plane beyond enqueue and claim.
References
Sources
- Redis — SCRIPT LOAD — load the script body once, before the pipeline of EVALSHA calls.
- Redis — EVALSHA — load-once, run-by-SHA dispatch the bulk verbs pipeline.
- Valkey — ZADD — the sorted-set re-score the batch lease extension performs per matching member.
- Valkey — Documentation — the substrate of record EchoMQ is backed by.
Related in this course
- Enqueue many — admit many in one flush.
- Bulk flows — compose many flows, fail-closed per flow.
- Batch lease extension — extend many leases under one clock read.
- The Queue — the pillar this module belongs to.
- redis-patterns · Reliable queues — the near side of the door.