R4.03.2 · dive 2
FIFO within a tier
Two jobs of the same priority share the same high bits. The tie is broken by the low bits —
the INCR pc arrival counter — so the earliest arrival carries the smaller
score.
A priority tier is a band of equal high bits. Inside that band the order is whatever the low bits say. This dive shows what fills the low bits — a monotonically rising arrival counter — and why it is the right tie-breaker. Toggle the counter on the right onto three same-priority jobs and watch their scores order them in the sequence they arrived.
Three same-priority jobs · the counter orders them FIFO
The tie a single field leaves
Score a job by its priority alone and every job in a tier gets the identical score. A sorted set keeps equal-score members in lexicographic order of the member, which for a job is its id — not its arrival order. So a priority-only score loses arrival order inside a tier: two jobs added in sequence can come out in id order, which is arbitrary.
ZADD prioritized 1 "job:zeta" # arrived first ZADD prioritized 1 "job:alpha" # arrived second # Equal scores → ordered by member: alpha before zeta — arrival order lost
A queue user expects first-in-first-out within a tier. A priority-only score cannot give it.
A priority-only score ties every job in a tier, and the set falls back to member order — the job id, not arrival order — so first-in-first-out is lost without a tie-breaker.
The counter breaks the tie
EchoMQ fills the low thirty-two bits with an arrival counter — the value of
INCR pc, taken inside the same script that scores the job.
-- inside getPriorityScore (included by addPrioritizedJob-9) — real local prioCounter = rcall("INCR", priorityCounterKey) return priority * 0x100000000 + prioCounter % 0x100000000
INCR returns the new value after incrementing, atomically. Each prioritized add reads a
fresh counter, one larger than the last. So within a tier the first add gets the smaller counter, the second a
larger one, and so on. The low bits now carry arrival order, and because they sit below the tier bits they break
the tie without disturbing it.
The result is FIFO within the tier. Three jobs of priority 1, added in order, take counters
1, 2, 3; their composite scores rise
in the same order; the set keeps them head-first by arrival. The hero figure toggles the
INCR pc counter on and off, and the priority-only mode shows the three tie and fall
back to member order.
The low bits are filled by INCR pc — a fresh, larger counter per add
— so same-tier jobs get rising scores in arrival order, breaking the tie without touching the tier.
Why the counter, not the clock
A timestamp could order arrivals too, but it ties when two jobs arrive in the same millisecond — the same
problem the delayed score solves with a low-bit shift. The INCR pc counter never ties:
it rises by exactly one per add, atomically, regardless of clock resolution. Two jobs added in the same
millisecond still get two different counters, so the within-tier order is total and deterministic. The counter
is a logical clock for arrivals, immune to wall-clock collisions.
Same millisecond, two jobs · counter vs timestamp
A timestamp ties when two jobs land on the same millisecond; the INCR pc
counter rises by one per add and never ties, so the within-tier order is total and deterministic.
In EchoMQ — the arrival counter, in real code
The counter key is EchoMQ.Keys.pc/1 → emq:{queue}:pc,
documented as the priority counter. It is passed as KEYS[9] via
Keys.pc(ctx) in EchoMQ.Scripts.add_prioritized_job/4, which runs
addPrioritizedJob-9.lua. Inside, getPriorityScore reads
rcall("INCR", priorityCounterKey) and folds it into the low thirty-two bits of the
score, then rcall("ZADD", prioritizedKey, score, jobId) adds the job.
-- EchoMQ.Scripts.add_prioritized_job/4 passes Keys.pc(ctx) as KEYS[9] (real) local prioCounter = rcall("INCR", priorityCounterKey) -- the next arrival number local score = priority * 0x100000000 + prioCounter % 0x100000000 rcall("ZADD", prioritizedKey, score, jobId) -- arrival order kept in the low bits
Every prioritized add takes its arrival number from the same counter, so the whole prioritized set shares one consistent arrival order. The counter is the difference between "this tier" and "this tier, in arrival order".
A textbook sorted set breaks an equal-score tie with lexicographic member order — arbitrary for a job id.
getPriorityScore breaks the within-tier tie with the
INCR pc arrival counter packed into the low thirty-two bits, so same-priority jobs
come out in the order they arrived: strict FIFO inside the tier, with no second structure.
A sorted set's equal-score tie falls back to member order; the INCR pc
counter in the low bits replaces that with arrival order, so a tier serves first-in-first-out.
References
The atomic increment behind the arrival counter, the write that places a job in its tier and arrival slot, the equal-score tie the counter removes, and the queue protocol EchoMQ ports.
Sources
- Redis — INCR — the atomic increment behind the arrival counter; returns the new value, one larger each call.
- Redis — ZADD — score a member by its composite score; the write that places a job in its tier and arrival slot.
- Redis — Sorted sets — equal-score members fall back to member order, the tie the counter removes.
- BullMQ — the queue protocol — the prioritized-jobs protocol EchoMQ ports, where the Lua scripts are the protocol.
Related in this course
- R4.03 · Priority with composite scores — the module hub.
- R4.03.1 · Packing two keys in one score — the previous dive: the tier in the high bits, the counter in the low.
- R4.03.3 · ZPOPMIN — the next dive: the smallest-score pop that returns the FIFO head.
- R4.01.1 · The score is the fire-time — the low-bit shift that orders within a millisecond.
- R4 · Time, Delay & Priority — the chapter.
- E4 · Groups — the dedicated EchoMQ course: intra-group priority in depth.