Reusing the delayed ZSET

When a job fails and has retries left, the worker calls EchoMQ.Scripts.retry_job/5, which runs retryJob-11.lua. That script's KEYS[7] is the delayed key — emq:{queue}:delayed, the exact set R4.01 schedules into. The failed job is re-added there, scored by its fire-time the way every delayed job is, with the score coming from the same getDelayedScore shift R4.01 walks: (timestamp + delay) × 0x1000, the fire-time shifted twelve bits so jobs due in the same millisecond stay ordered.

The delay in that fire-time is the backoff delay — computed first by EchoMQ.Backoff.calculate/4 in Elixir, then handed to the reschedule. So the retry sits in the delayed set indistinguishable from a freshly-scheduled deferred job: same key, same score shape, same ordering. The set does not separate one member as a retry and another as a first-time schedule. To the sorted set they are all jobs scored by when they are due.

retryJob-11.lua does not only re-add the job — it includes the promote sweep. The script carries [INCLUDED: includes/promoteDelayedJobs.lua], and that included function is the same one R4.01's promotion dive teaches.

-- promoteDelayedJobs (included by retryJob-11.lua) — real
local jobs = rcall("ZRANGEBYSCORE", delayedKey, 0, (timestamp + 1) * 0x1000 - 1, "LIMIT", 0, 1000)
if (#jobs > 0) then
    rcall("ZREM", delayedKey, unpack(jobs))    -- then ZADD each onto the wait / prioritized list
end

It ranges the delayed key from 0 up to (timestamp + 1) × 0x1000 - 1 — every job whose shifted fire-time is at or below the current millisecond — removes them with ZREM, and moves them onto the wait or prioritized list. The retry re-enters at a future score and is swept back by exactly this function when the clock reaches it. There is no retry-specific sweep; the retry is promoted by the same machinery that promotes every delayed job, invoked as promoteDelayedJobs(KEYS[7], ...) inside the retry script.

The whole module reduces to this: R4.01 owns the delayed set, the score shift, and the promote sweep. R4.04 reuses all three. What R4.04 adds is upstream of the set — the computation of which delay to put in the score, done by EchoMQ.Backoff (the exponential doubling, the jitter). What R4.04 does not add is any new key, command, or sweep. The reschedule is a ZADD to emq:{queue}:delayed; the recovery is promoteDelayedJobs; both are R4.01's.

This is why the pattern is economical. A queue that already has a delayed queue has a retry mechanism for free — it needs only a formula to size the delay and an attempts counter to bound the loop. The hard part (a time-ordered set with an efficient due-range read) was built once, in R4.01, and serves both scheduled work and retries. The interactive below puts a first-time schedule and a retry in the same set and sweeps them with one clock.

The reuse is real code in echo/apps/echomq. EchoMQ.Scripts.retry_job/5 runs retryJob-11.lua; KEYS[7] is the delayed key from EchoMQ.Keys.delayed/1 (emq:{queue}:delayed). The script re-adds the failed job there at the backoff score and then calls the included promoteDelayedJobs(KEYS[7], ...) — the same sweep addDelayedJob-6.lua and the worker scripts include. The backoff delay comes from EchoMQ.Backoff; the key, the score shift, and the sweep are R4.01's.

-- retryJob-11.lua — re-add at the backoff score, then sweep (real shape)
-- KEYS[7] = emq:{queue}:delayed
ZADD KEYS[7] backoffScore jobId          -- backoffScore = (now + backoff) × 0x1000
promoteDelayedJobs(KEYS[7], markerKey, target, ...)  -- the same included sweep R4.01 teaches

The re-add at the backoff fire-time, the due-sweep that brings the retry back, the one sorted set both uses share, the queue protocol EchoMQ ports, and the direction EchoMQ 2.0 takes native.