jonnifyredis

R4.02.2 · dive 2

The upsert, no duplicates

A reboot re-registers the same scheduler. The ZSCORE-then-ZADD upsert keeps exactly one registry entry per scheduler key — no duplicate recurring job on boot.

A service that owns recurring schedules registers them at startup, and again on every restart. The upsert probes the registry with ZSCORE before storing with ZADD; re-adding the same member rewrites its score rather than appending a row. Run two boots over the same scheduler key on the right and watch the registry stay at one entry.

Two boots, one scheduler key · upsert stays at one entry, a naive append reaches two

emq:{queue}:repeat (ZSET) — after boot 1 and boot 2 sched:reports score = next-run millis (boot 1, then rewritten by boot 2) sched:reports score = next-run millis (boot 2 — a SECOND row) upsert · ZSCORE found boot 1 → ZADD rewrote it registry size: 1 entry — no duplicate

The naive add duplicates

A naive registration appends. Boot once: the registry has one entry. The danger on a re-registration is a stale second future run — a re-registration that recomputed a different next-run time can leave the previous registration's delayed job orphaned in :delayed, so the schedule fires twice around the boundary. The probe is what finds and reconciles that stale entry before the new one is stored.

Switch the figure to the naive append: the second boot adds a second row, and the scheduler now produces two recurring jobs on the same cadence. The upsert is the guard that keeps the registry at one entry per scheduler key and removes the orphaned delayed job.

A naive re-registration can leave a second, orphaned future run; the upsert is the guard that keeps one entry per scheduler key and reconciles the stale delayed job.

ZSCORE probes, ZADD stores

The upsert is two commands. First the probe: ZSCORE repeatKey customKey returns the prior next-run score if the scheduler key is already registered, or false if it is new. If a prior score is found, the script reconciles the stale delayed entry from the previous registration before storing the new one.

Then the store: ZADD repeatKey nextMillis customKey. Because the member is the scheduler's own key, the ZADD either inserts a new row or rewrites the existing row's score — never two rows for one scheduler. The registry's cardinality stays at one entry per scheduler key across every boot.

ZSCORE
ZSCORE repeatKey customKey — the probe; returns the prior next-run score or false.
reconcile
If a prior score exists, remove the stale delayed job from the previous registration before re-storing.
ZADD
ZADD repeatKey nextMillis customKey — the store; the same member rewrites rather than appends.

Probe with ZSCORE, reconcile any stale delayed job, store with ZADD; the same member keeps the registry at one entry per scheduler key.

In EchoMQ — the probe in addRepeatableJob

The real upsert lives in addRepeatableJob-2.lua. The probe runs before the store reconciles whatever the previous registration left in :delayed.

-- addRepeatableJob-2.lua (real)
local prevMillis = ZSCORE repeatKey customKey   -- was this scheduler already registered?
if prevMillis then
  local delayedJobId = "repeat:"..customKey..":"..prevMillis
  -- reconcile the stale delayed entry from the prior next-run, then re-store
end

-- storeRepeatableJob (called below) — the store
ZADD repeatKey nextMillis customKey        -- one row per scheduler key

prevMillis is the prior next-run timestamp. A present prevMillis means the scheduler key was already in the registry, so the script removes the stale delayed job the previous registration scheduled (removeJob + ZREM delayedKey delayedJobId) before storeRepeatableJob rewrites the registry entry. The result is exactly one :repeat entry per scheduler key and no orphaned duplicate run. The newer scheduler API does the same thing keyed by a scheduler id — storeJobScheduler(...) in addJobScheduler-11.lua runs rcall("ZADD", repeatKey, nextMillis, schedulerId) on the same registry sorted set.

The probe result · ZSCORE returns false (new) or a prior score (re-registered)

ZSCORE emq:{queue}:repeat sched:reports → false (new) ZADD inserts sched:reports → one new row first registration · ZSCORE false · ZADD inserts registry size after: 1 entry
the pattern

A textbook delayed queue adds a fire-time and forgets it; a scheduler must survive re-registration without re-firing. Probe the registry, then store: the probe is the idempotence and the same member rewrites rather than appends.

in EchoMQ

addRepeatableJob-2.lua runs local prevMillis = rcall("ZSCORE", repeatKey, customKey), reconciles the stale delayed job if present, then storeRepeatableJob runs rcall("ZADD", repeatKey, nextMillis, customKey) — one entry per scheduler key, no duplicate on boot.

The pattern says probe then store so a re-registration rewrites rather than appends; in EchoMQ the probe is ZSCORE and the store is ZADD, keeping the :repeat registry at one entry per scheduler.

Door · not depth The full scheduler-vs-delayed reconciliation — how a re-registration that changes the cadence retires the old delayed job and how an overlapping boot is serialized — is the scheduler subsystem the dedicated EchoMQ course covers at E6 · Lifecycle controls. This dive teaches the ZSCORE/ZADD upsert; that course teaches the reconciliation around it.

References

The probe that reads the prior score, the store that rewrites rather than appends, the sorted set whose member uniqueness makes the upsert work, and the queue protocol whose registration path EchoMQ ports.

Sources

Related in this course