R4 · Time, Delay & Priority
The sorted set as a clock
One data structure, read two ways: score a job by a future time and the set is a timer wheel; score it by a priority and the set is a ladder.
A sorted set keeps its members ordered by a numeric score, and that is the whole chapter. Score a job by its fire-time and a range query pulls out everything now due — a delayed queue. Pack a priority and an arrival counter into one score and the smallest entry is the next to run — a priority queue with FIFO inside each tier. Score by next-run time and the set is a scheduler registry. EchoMQ uses exactly this: emq:{queue}:delayed scores by timestamp × 0x1000, emq:{queue}:prioritized by priority × 2³² + arrival, and EchoMQ.Backoff reschedules a failed job onto the same delayed set. It stands on R3's reliable queue.
Overview
R3 makes a job reliable: it can be delivered now, once, and recovered if a worker dies. It cannot yet be held for later, recurred on a schedule, ordered by importance, or backed off after a failure. Every one of those is a property of when and in what order a job runs — and every one is a sorted set whose score carries the answer.
A Redis sorted set (ZSET) stores members ordered by a numeric score, with range queries by score and a pop of the smallest. Put a future timestamp in the score and ZRANGEBYSCORE 0 now returns the jobs whose time has come; the textbook delayed queue is exactly that. Put a composite of priority and arrival in the score and ZPOPMIN returns the most important job, oldest-first within a tier. The commands never change; the meaning of the score does.
EchoMQ runs both readings in real code.
addDelayedJob-6.lua
scores the delayed set by (timestamp + delay) × 0x1000 — the millisecond
shifted left twelve bits so a low-bit discriminator keeps a stable order inside one millisecond — and a
promoteDelayedJobs sweep
ranges the due head to the wait list. getPriorityScore packs
priority × 0x100000000 + INCR pc into one double so the priority set pops by
tier then arrival. The same machinery underlies schedulers, backoff, and leaderboards.
One sorted set, two readings · a timer wheel and a priority ladder
A delayed queue and a priority queue are the same sorted set under two readings of the score — ZADD to insert, a range or a pop to read.
Why & when
Reach for the sorted set whenever a job's time or its order matters — not only whether it runs, but when and after which others. Each need below maps to one score reading, and the chapter is the set of those readings.
Run a job later
A reminder, a deferred email, a cooldown — score the job by its fire-time and sweep the due head with a range query.
Recur a job without duplicates
A nightly digest, a polling task — register it in a scheduler set keyed by next-run time, upserted so a reboot adds no duplicate.
Order a queue by priority
Paid before free, urgent before routine — pack priority and arrival into one score so the queue is strict by tier, FIFO inside it.
Retry after a failure
Back off so a failing dependency is not hammered — reschedule onto the delayed set with an exponential delay plus jitter.
Every technique in this chapter answers one question about a job — run it when, recur it how often, run it before which others, retry it after how long — and every answer is a sorted-set score.
The patterns
Three deep dives carry the chapter's orientation — the sorted set as one structure read two ways, the score as the whole semantic axis, and where the chapter goes next. Each is a full lesson grounded in EchoMQ's real time and priority sets. The granular module ladder below takes each pattern in depth, one hub plus three dives per module.
The sorted set as a clock
One ZSET, two readings: a timer wheel (:delayed, score = fire-time) and a priority ladder (:prioritized, score = composite). The member is constant; the score is the meaning.
Score as meaning
The score is the semantics: fire-time (ts×0x1000), composite priority (priority×2³²+pc), and next-run millis (:repeat) — same commands, three meanings.
The road ahead
Where the chapter goes: the arc R4.01→R4.06, and the door into the living EchoMQ course's scheduler subsystem.
The granular modules — each a hub plus three dives, closing with the workshop capstone:
The delayed queue
Score a job by its fire-time, sweep by score: the :delayed ZSET and the promoteDelayedJobs range. Dives: score = fire-time · ZRANGEBYSCORE promotion · the next-wake computation.
Schedulers & repeatable jobs
Recurring jobs via cron or interval on the :repeat ZSET, upserted so a boot adds no duplicate. Dives: cron vs interval · upsert · start-to-start cadence.
Priority with composite scores
Pack priority and arrival into one score: getPriorityScore = priority×0x100000000+pc. Dives: packing two keys in one score · FIFO-within-tier · ZPOPMIN.
Backoff & retry
Exponential backoff with jitter on the same delayed set: EchoMQ.Backoff = base×2ⁿ⁻¹. Dives: exponential backoff · jitter · the delayed-ZSET reuse.
Leaderboards
Real-time rankings on the same ZSET machinery, over Portal progress. Dives: ZADD/ZRANK · top-N & around-me · the score-update path.
Workshop
The capstone: schedule Portal's notification and digest jobs on the :delayed/:repeat sets.
How to apply
The choice is always the same one question: what does the job's score need to mean? Name the scheduling need and the score reading — and the real EchoMQ set that implements it — follows.
Name the scheduling need · get the score reading
There is no separate data structure for time and for priority — only the sorted set, and the score you choose to give it, each one a real set in EchoMQ's scheduler.
The road ahead
R4 adds the time and order axes to R3's reliable job. With scheduling in hand, the later chapters are further surfaces over the same jobs — the durable event log, flow control under load, the data model, and operating the tier. They build on the queue and its schedule rather than replacing them.
R5 · Streams & Events
The durable log — observe and replay the lifecycle of the jobs R3/R4 schedule.
R6 · Flow Control
Staying stable under load — rate-limit, batch, and bound concurrency, with priority as the fairness lever.
R7 · Data Modeling
How data lives in RAM — the job record, the read-models, and the leaderboard sorted sets.
R8 · Production & Operations
Running the tier at scale — operate the scheduled, prioritized queue in production.
Streams, flow control, the data model, and operations are all surfaces over the one scheduled, prioritized queue R4 builds on top of R3.
Up next
With time and priority on the queue, the next surfaces over its jobs follow — the durable event log, flow control under load, the data model, and operations.
The door
This chapter teaches the sorted-set time and priority patterns and proves each with one real EchoMQ excerpt. The scheduler subsystem that runs them — the timezone and DST rules of a cron schedule, the guard that holds a job while its slot is busy, the migration of legacy repeatable jobs, and the scheduling of parent-child job flows — is the subject of the dedicated EchoMQ course.
→ EchoMQ. The EchoMQ course is the living companion depth course: it teaches the polyglot job-queue protocol in depth and tracks the EchoMQ build rung by rung. Per its cross-link map, R4 opens onto E6 · Lifecycle controls — the TTL clock, the delay and promote path, and the retry/checkpoint machinery — and E4 · Groups, where intra-group priority and the control plane live. The course home is /echomq.
R4's patterns are transferable Redis techniques; the EchoMQ course owns the scheduler implementation depth behind them.
References
Sources
- Redis — Sorted sets — the data type that orders members by score, the substrate for the whole chapter.
- Redis — ZRANGEBYSCORE — the range query that sweeps the due head of a delayed set.
- Redis — ZPOPMIN — the pop of the smallest score, the priority queue's read.
- BullMQ — Delayed & prioritized jobs — the protocol EchoMQ ports across runtimes.
Related in this course
- R3 · Reliable Queues — the reliable job this chapter schedules and orders.
- R3.05 · Blocking vs polling — the marker the delayed set wakes a worker through.
- R0.2 · Redis under Portal — the EchoMQ bus these patterns ground in.
- /elixir · The GenServer — the runtime process the scheduler's timers run inside.