The sorted set as a clock

The classic delayed-queue pattern scores each task by the Unix timestamp at which it should run, and stores it in a sorted set. The tasks order themselves by execution time, earliest first, with no extra bookkeeping: the member is the task id, the score is the timestamp it is due.

A worker polls for tasks whose time has passed with a range query bounded above by the current clock. ZRANGEBYSCORE delayed_queue -inf <now> LIMIT 0 10 returns up to ten tasks with scores at or below now; the -inf lower bound means no floor. Because many workers may poll the same set at once, a worker claims each task atomically before processing it. ZREM returns 1 when this call removed the member — the caller won the claim — or 0 when another worker already removed it; a worker processes a task only on a 1.

# schedule a task for a future time
ZADD delayed_queue 1706649000 "task:abc123"   # score = the Unix timestamp it is due

# poll the due head, then claim each atomically
ZRANGEBYSCORE delayed_queue -inf now LIMIT 0 10 # up to 10 tasks at or below now
ZREM delayed_queue "task:abc123"           # 1 = this worker claimed it; 0 = lost the race

Constant polling of an empty set wastes work. The avoidance is to read the head and sleep until it is due: ZRANGE delayed_queue 0 0 WITHSCORES returns the earliest task and its scheduled time, so a worker sleeps until that time (or a bounded maximum) instead of spinning. A failed task reschedules itself by re-adding with a future score — a retry is a delayed task like any other. Slide the clock in the figure above: every job at or left of the now line is in the ZRANGEBYSCORE 0 now due set, and the first job to its right is the next wake the head returns.

The same set shape carries a second pattern. Leave the member as the job id, but score it by a composite of two numbers packed into one: a priority tier in the high bits and an arrival counter in the low bits, so the score is priority × 0x100000000 + arrival. Now the sorted set is a priority queue: the smallest score is the most important job, and within one tier the smallest score is the earliest arrival. A pop of the smallest, ZPOPMIN, returns the next job in priority-then-FIFO order.

Packing two fields into one score is what lets a single read command do the ordering. The priority tier occupies the high 32 bits, so any job in a more important tier sorts ahead of every job in a less important one. The arrival counter occupies the low 32 bits, so two jobs in the same tier break their tie by arrival order — first in, first out, inside the tier. One number, two meanings, read by one ZPOPMIN. Toggle the reading below: the same six jobs reorder, because the score is computed and read differently, while the structure underneath stays one sorted set.

# the composite priority score — high bits = tier, low bits = arrival
score = tier * 0x100000000 + arrival   # 0x100000000 = 2^32
ZADD prioritized score jobId          # same insert as the delayed set
ZPOPMIN prioritized                  # smallest score = top tier, oldest-first within it

The textbook pattern scores by a raw Unix timestamp. EchoMQ keeps the structure and sharpens the score. Its delayed set is EchoMQ.Keys.delayed/1 → emq:{queue}:delayed, scored by (timestamp + delay) × 0x1000 — the fire-time millisecond shifted left twelve bits so the low twelve bits can pack an ordering discriminator within a single millisecond (addDelayedJob-6.lua's getDelayedScore, minScore = delayedTimestamp * 0x1000). The recovery is exact, not lossy: getNextDelayedTimestamp reads the head and returns nextTimestamp / 0x1000, dividing the twelve-bit shift back out to recover the real millisecond.

-- addDelayedJob-6.lua — getDelayedScore (real)
local delayedTimestamp = timestamp + delay
local minScore = delayedTimestamp * 0x1000   -- ms shifted left 12 bits
rcall("ZADD", delayedKey, minScore, jobId)   -- emq:{queue}:delayed
-- getNextDelayedTimestamp: nextTimestamp / 0x1000 recovers the real ms

Where the textbook worker sleeps and re-polls, EchoMQ runs a sweep. The included promoteDelayedJobs Lua function ranges the due head — ZRANGEBYSCORE delayedKey 0 (timestamp + 1) * 0x1000 - 1 LIMIT 0 1000 — then ZREMs each member and moves it to the wait list. The priority reading is EchoMQ.Keys.prioritized/1 → emq:{queue}:prioritized, scored by getPriorityScore = priority × 0x100000000 + (INCR pc) % 0x100000000 — the priority tier in the high 32 bits, the pc arrival counter in the low 32 bits — and read by ZPOPMIN. Two named sets, one sorted-set shape, two readings of the score.

The data type both readings stand on, the single insert they share, and the two reads — the range over the due head and the pop of the smallest — plus the queue protocol EchoMQ ports.