jonnifyredis

R3.05.3 · dive 3

The marker wake-up

EchoMQ blocks on a dedicated marker ZSET, not on the work list. The marker is a doorbell — the producer rings it, the parked worker wakes.

The worker parks on BZPOPMIN emq:{queue}:marker; the producer, when it adds a job, runs ZADD emq:{queue}:marker 0 "0", which makes the parked block return. One ZADD wakes one BZPOPMIN. Press the button on the right to add a job and ring the marker, and watch the parked worker wake and fetch.

The doorbell handshake · park on BZPOPMIN, ring with ZADD, wake, fetch

worker (blocking connection) BZPOPMIN · parked emq:{queue}:marker (ZSET) empty parked on fetch from wait the worker is parked · the marker is empty

The marker is a ZSET

EchoMQ.Keys.marker/1 builds emq:{queue}:marker, documented as "the marker sorted set (for blocking operations)". It is a separate key from the work lists emq:{queue}:wait and emq:{queue}:active. The worker does not block on the work list; it blocks on the marker, and fetches the job from the work lists once woken. Splitting the signal from the work keeps the blocking connection parked on one small key and the fetch on the main connection.

marker
emq:{queue}:marker — a ZSET, the signal a worker parks on with BZPOPMIN. Built by EchoMQ.Keys.marker/1.
doorbell
The marker carries no job data; its presence is the whole signal. The producer rings it; one ring wakes one parked worker.
wait
The work list the woken worker fetches the job from, on its main connection — separate from the marker.

The marker is a dedicated ZSET, separate from the work lists. The worker parks on it; the job lives in wait, fetched once the marker rings.

The handshake

The wake-up is four steps across the worker side and the producer side.

# 1. worker — park on the marker (blocking connection)
BZPOPMIN emq:{queue}:marker 30

# 2. producer — add the job, then ring the doorbell
LPUSH emq:{queue}:wait <jobId>
ZADD  emq:{queue}:marker 0 "0"      # addBaseMarkerIfNeeded

# 3. worker — the ZADD woke the parked block
BZPOPMIN returns [marker, "0", 0]    # do_wait_for_job/3 → :job_available

# 4. worker — fetch the job from wait/active (main connection), then run it

Step 1 parks the worker. Step 2 adds the job and rings the marker — addBaseMarkerIfNeeded runs ZADD emq:{queue}:marker 0 "0" (verified in addStandardJob-9.lua and drain-5.lua). Step 3 is the wake: the ZADD gives the marker a member, so the parked BZPOPMIN pops it and returns; do_wait_for_job/3 reads that as :job_available. Step 4 fetches the job and runs the lifecycle. The marker's value is a constant "0" — its presence is the whole signal.

Park on BZPOPMIN, ring with ZADD marker 0 "0", the block returns :job_available, fetch from wait — four steps, and the marker carries no data but its presence.

One ring, one worker

BZPOPMIN pops the lowest-scored member and removes it. A single ZADD adds one member, so a single ZADD wakes exactly one parked BZPOPMIN; any other workers parked on the same marker stay parked, waiting for the next ring. This is fan-out fairness: one added job wakes one worker, not a thundering herd. Set how many workers are parked below and ring the marker once — exactly one wakes.

Fan-out fairness · N workers parked, one ZADD — how many wake

3
workers parked on BZPOPMIN emq:{queue}:marker —

One ZADD adds one member, so one ring wakes exactly one parked BZPOPMIN; the rest stay parked — fan-out fairness, no thundering herd.

In EchoMQ — the doorbell, in real code

The whole handshake is real code in echo/apps/echomq. The worker's blocking-fetch internals EchoMQ.Worker.wait_for_job/2 → do_wait_for_job/3 park on BZPOPMIN emq:{queue}:marker timeout on the dedicated blocking connection and return :job_available or :timeout. The producer's addBaseMarkerIfNeeded runs ZADD emq:{queue}:marker 0 "0" whenever a job is added, in addStandardJob-9.lua, drain-5.lua, and the other add scripts.

# addBaseMarkerIfNeeded.lua (included by addStandardJob-9 / drain-5) — ring the doorbell (real)
ZADD emq:{queue}:marker 0 "0"   -- adding a job ZADDs the marker → the blocked BZPOPMIN returns

The Go port's empty-queue worker loop does not park on this marker — it sleeps a fixed interval (time.Sleep(100 * time.Millisecond) / time.Sleep(time.Second) in apps/echomq-go/pkg/echomq/worker_impl.go), the cross-runtime contrast: a poll cadence rather than a marker block. The base marker is scored 0; a delayed job rings a marker scored by its timestamp instead, which is the time-and-delay chapter's territory, not this one.

the pattern

Block on a signal and ring it when work arrives, so the parked worker wakes the instant a job is added; one ring wakes one worker, so a burst of additions wakes a matching number of workers, one each.

in EchoMQ

The signal is the emq:{queue}:marker ZSET, the block is BZPOPMIN in do_wait_for_job/3, and the ring is the producer's ZADD marker 0 "0" (addBaseMarkerIfNeeded). The Go port's time.Sleep loop is the contrast.

The pattern says block on a signal and ring it when work arrives; in EchoMQ the signal is the marker ZSET, the block is BZPOPMIN, and the ring is the producer's ZADD.

Door · not depth The full worker fetch loop and the polyglot concurrency models that govern how each runtime parks and wakes are the subject of the dedicated EchoMQ course. The worker fetch loop is E2 · the engine; the marker and heartbeat coordination across a worker pool is E6 · the lifecycle. This dive teaches the marker wake-up; that course teaches the engine that rings and parks on it.

References

The marker pop the worker parks on, the ZADD that rings it, the sorted set behind it, and the queue protocol whose doorbell EchoMQ ports.

Sources

  • Redis — BZPOPMIN — block on a sorted set until a member is added; the marker pop the worker parks on.
  • Redis — ZADD — add a member to a sorted set; the producer's ZADD marker that wakes a blocked worker.
  • Redis — Sorted sets — the ZSET data type behind the marker and the BZPOPMIN pop.
  • BullMQ — the queue protocol — the marker / BZPOPMIN doorbell EchoMQ ports, where the Lua scripts are the protocol.

Related in this course