jonnifyredis

R4 · orientation

Score as meaning

The sorted set is one mechanism. The score is the whole semantic axis — the same number, read three ways.

A sorted set stores members ordered by a floating-point score. The textbook delayed queue uses the plainest score there is: a Unix timestamp. EchoMQ keeps the same set and the same commands, but splits the score's bits so one double carries two facts at once — a priority tier in the high 32 bits and an arrival counter in the low 32, or a fire-time in the high bits and a 12-bit discriminator below. Score a job by fire-time and the set is a delayed queue; by composite priority and it is a priority ladder; by next-run time and it is a scheduler registry. Set the priority and arrival on the right and watch the actual score the writer would store.

The composite-score decoder · priority × 2³² + arrival

2
17
high 32 bits · priority tier low 32 bits · arrival counter tier 2 arrival 17 score = tier × 0x100000000 + arrival score = 8589934609

The score is a number that sorts

A sorted set stores members ordered by a floating-point score. The textbook delayed queue uses the simplest score there is — the Unix timestamp at which a task should run. The member is the task id, the score is the fire-time, and the set stays ordered by fire-time with no extra bookkeeping:

# the delayed-queue pattern — score is the fire-time, member is the task id
ZADD delayed_queue 1706649000 "task:abc123"      # schedule for that Unix second
ZRANGEBYSCORE delayed_queue -inf 1706648500 LIMIT 0 10   # read everything due now
ZREM delayed_queue "task:abc123"            # atomic claim: 1 = this worker won

Nothing in Redis is told what the score “is” — it is a number, and the set sorts by it. The meaning lives entirely in how the writer chooses the number and how the reader interprets the order. That freedom is the lever this dive turns. Because the score is an arbitrary double, a writer can pack more than one fact into it, as long as the resulting order is still the order the reader needs.

The hero figure runs exactly that move. The composite priority EchoMQ stores on its emq:{queue}:prioritized set is priority × 0x100000000 + arrival — the tier sits in the high 32 bits, the arrival counter in the low 32. Slide the tier and the arrival number and the figure computes the actual score a writer would ZADD, and reads back where that job lands in the order.

The sorted set does not store time or priority — it stores one number per member and keeps the set ordered by it. The number is the meaning.

Two meanings packed into one double

EchoMQ scores its :prioritized set with a composite priority. The Lua function getPriorityScore — verbatim in addPrioritizedJob-9.lua, addDelayedJob-6.lua, changePriority-7.lua and three more — computes:

# getPriorityScore — pack a tier and an arrival number into one score
prioCounter = rcall("INCR", priorityCounterKey)   # the :pc arrival counter
score       = priority * 0x100000000 + prioCounter % 0x100000000
# 0x100000000 = 2^32 = 4294967296

Multiplying the priority by 0x100000000 (2³² = 4294967296) shifts it into the high 32 bits of the score; the arrival counter — the next value of INCR :pc — occupies the low 32 bits. One double now carries two facts at once:

high 32 bits
the priority tier — the primary sort key. A lower tier value is more urgent, and ZPOPMIN pops the smallest score first, so tier 1 always outranks tier 2.
low 32 bits
the arrival number — the tiebreaker. Within a tier, the earlier-enqueued job has the smaller arrival counter, so the smaller score, so it is served first. Strict priority across tiers, FIFO within a tier — from one number.

The delayed set uses the same trick at a different scale. getDelayedScore (addDelayedJob-6.lua) scores a delayed job by delayedTimestamp × 0x1000, where delayedTimestamp = timestamp + delay. 0x1000 is 4096 = 2¹², so the fire-time in milliseconds sits in the high bits and a 12-bit discriminator sits below — two jobs due in the same millisecond can still be ordered. To recover the real millisecond, getNextDelayedTimestamp reads the head and returns nextTimestamp / 0x1000, dividing the 12-bit shift back out.

The figure below stores one fixed example job three ways. Pick a reading and it computes the actual score, the write command, and the read command — the same job, three meanings.

Three readings · the same job, stored three ways

emq:{q}:delayed fire-time · (now + delay) × 4096 — emq:{q}:prioritized priority · tier × 2³² + arrival — emq:{q}:repeat next-run · nextRunMs (upsert) —

A 12-bit shift packs a discriminator under a millisecond; a 32-bit shift packs an arrival counter under a tier. The same idea at two scales — one number, two facts.

Three sets, one mechanism

The third reading is the scheduler registry. EchoMQ's :repeat set (EchoMQ.Keys.repeat/1 → "#{base}:repeat") is a ZSET scored by the next-run millisecond. storeRepeatableJob (addRepeatableJob-2.lua, addJobScheduler-11.lua) runs ZADD repeatKey nextMillis customKey, and upserts by reading the prior score with ZSCORE repeatKey customKey and replacing it — so a reboot that re-registers the same recurring job does not duplicate it. (The keys.ex doc-comment that calls :repeat a “hash” is a source defect; the code uses ZADD/ZSCORE, which are ZSET commands.)

Three sorted sets, one mechanism. The commands stay constant — ZADD to write, ZRANGEBYSCORE / ZPOPMIN / ZSCORE / ZRANGE to read — and the score is the whole semantic axis.

the pattern

A sorted set orders members by a floating-point score. The member is the task id; the score is whatever number a writer encodes and a reader interprets.

in EchoMQ

:delayed scores by (timestamp + delay) × 0x1000; :prioritized by getPriorityScore (priority × 0x100000000 + INCR :pc); :repeat by the next-run millis, upserted with ZADD/ZSCORE.

To read an EchoMQ set is to read what its score means — fire-time, priority, or schedule. One mechanism, three readings, all carried by a single number.

A door, not a depth The full scheduler subsystem — the delay, promote and retry scripts, the backoff math, and the repeatable registry — is the dedicated EchoMQ course, a living agile-spec course that tracks the build rung by rung. Per its cross-link map, R4 opens onto E6 · Lifecycle controls (the delay, promote, retry and TTL depth) and E4 · Groups (intra-group priority and the control plane). This dive teaches the score; that course teaches the engine that writes it.

References

The sorted-set data type every reading shares, the one write command behind all three scores, the score-read that powers the :repeat upsert, the min-pop that turns a composite score into a ladder, and the protocol EchoMQ ports.

Sources

  • Redis — Sorted sets — members ordered by a floating-point score; the substrate every reading shares.
  • Redis — ZADD — add or update a member's score; the one write behind all three readings, and the :repeat upsert.
  • Redis — ZSCORE — read a member's current score; the upsert check that lets :repeat replace a prior next-run rather than duplicate it.
  • Redis — ZPOPMIN — pop the lowest-scored member; the read that turns the composite-priority score into a strict-then-FIFO ladder.
  • BullMQ — the reliable-queue protocol EchoMQ ports, where the composite priority and the × 0x1000 delayed score originate.

Related in this course