R3.05.3 · dive 3
The marker wake-up
EchoMQ blocks on a dedicated marker ZSET, not on the work list. The marker is a doorbell — the producer rings it, the parked worker wakes.
The worker parks on BZPOPMIN emq:{queue}:marker; the producer, when it
adds a job, runs ZADD emq:{queue}:marker 0 "0", which makes the parked block return.
One ZADD wakes one BZPOPMIN. Press the button on the right to
add a job and ring the marker, and watch the parked worker wake and fetch.
The doorbell handshake · park on BZPOPMIN, ring with ZADD, wake, fetch
The marker is a ZSET
EchoMQ.Keys.marker/1 builds emq:{queue}:marker, documented as
"the marker sorted set (for blocking operations)". It is a separate key from the work lists
emq:{queue}:wait and emq:{queue}:active. The worker does not
block on the work list; it blocks on the marker, and fetches the job from the work lists once woken. Splitting
the signal from the work keeps the blocking connection parked on one small key and the fetch on the main
connection.
- marker
emq:{queue}:marker— a ZSET, the signal a worker parks on withBZPOPMIN. Built byEchoMQ.Keys.marker/1.- doorbell
- The marker carries no job data; its presence is the whole signal. The producer rings it; one ring wakes one parked worker.
- wait
- The work list the woken worker fetches the job from, on its main connection — separate from the marker.
The marker is a dedicated ZSET, separate from the work lists. The worker parks on it; the job lives in
wait, fetched once the marker rings.
The handshake
The wake-up is four steps across the worker side and the producer side.
# 1. worker — park on the marker (blocking connection) BZPOPMIN emq:{queue}:marker 30 # 2. producer — add the job, then ring the doorbell LPUSH emq:{queue}:wait <jobId> ZADD emq:{queue}:marker 0 "0" # addBaseMarkerIfNeeded # 3. worker — the ZADD woke the parked block BZPOPMIN returns [marker, "0", 0] # do_wait_for_job/3 → :job_available # 4. worker — fetch the job from wait/active (main connection), then run it
Step 1 parks the worker. Step 2 adds the job and rings the marker —
addBaseMarkerIfNeeded runs ZADD emq:{queue}:marker 0 "0"
(verified in addStandardJob-9.lua and drain-5.lua). Step 3 is
the wake: the ZADD gives the marker a member, so the parked
BZPOPMIN pops it and returns; do_wait_for_job/3 reads that as
:job_available. Step 4 fetches the job and runs the lifecycle. The marker's value is a
constant "0" — its presence is the whole signal.
Park on BZPOPMIN, ring with ZADD marker 0 "0", the
block returns :job_available, fetch from wait — four steps,
and the marker carries no data but its presence.
One ring, one worker
BZPOPMIN pops the lowest-scored member and removes it. A single
ZADD adds one member, so a single ZADD wakes exactly one
parked BZPOPMIN; any other workers parked on the same marker stay parked, waiting for
the next ring. This is fan-out fairness: one added job wakes one worker, not a thundering herd. Set how many
workers are parked below and ring the marker once — exactly one wakes.
Fan-out fairness · N workers parked, one ZADD — how many wake
One ZADD adds one member, so one ring wakes exactly one parked
BZPOPMIN; the rest stay parked — fan-out fairness, no thundering herd.
In EchoMQ — the doorbell, in real code
The whole handshake is real code in echo/apps/echomq. The worker's blocking-fetch
internals EchoMQ.Worker.wait_for_job/2 →
do_wait_for_job/3 park on BZPOPMIN emq:{queue}:marker timeout
on the dedicated blocking connection and return :job_available or
:timeout. The producer's addBaseMarkerIfNeeded runs
ZADD emq:{queue}:marker 0 "0" whenever a job is added, in
addStandardJob-9.lua, drain-5.lua, and the other add
scripts.
# addBaseMarkerIfNeeded.lua (included by addStandardJob-9 / drain-5) — ring the doorbell (real) ZADD emq:{queue}:marker 0 "0" -- adding a job ZADDs the marker → the blocked BZPOPMIN returns
The Go port's empty-queue worker loop does not park on this marker — it sleeps a fixed interval
(time.Sleep(100 * time.Millisecond) / time.Sleep(time.Second)
in apps/echomq-go/pkg/echomq/worker_impl.go), the cross-runtime contrast: a poll cadence
rather than a marker block. The base marker is scored 0; a delayed job rings a marker
scored by its timestamp instead, which is the time-and-delay chapter's territory, not this one.
Block on a signal and ring it when work arrives, so the parked worker wakes the instant a job is added; one ring wakes one worker, so a burst of additions wakes a matching number of workers, one each.
The signal is the emq:{queue}:marker ZSET, the block is
BZPOPMIN in do_wait_for_job/3, and the ring is the producer's
ZADD marker 0 "0" (addBaseMarkerIfNeeded). The Go port's
time.Sleep loop is the contrast.
The pattern says block on a signal and ring it when work arrives; in EchoMQ the signal is the marker
ZSET, the block is BZPOPMIN, and the ring is the producer's
ZADD.
References
The marker pop the worker parks on, the ZADD that rings it, the sorted set behind it, and the queue
protocol whose doorbell EchoMQ ports.
Sources
- Redis — BZPOPMIN — block on a sorted set until a member is added; the marker pop the worker parks on.
- Redis — ZADD — add a member to a sorted set; the producer's
ZADD markerthat wakes a blocked worker. - Redis — Sorted sets — the ZSET data type behind the marker and the
BZPOPMINpop. - BullMQ — the queue protocol — the marker /
BZPOPMINdoorbell EchoMQ ports, where the Lua scripts are the protocol.
Related in this course
- R3.05 · Blocking vs polling — the module hub.
- R3.05.1 · The busy-poll cost — the polling loop the marker block replaces.
- R3.05.2 · Blocking pop — the blocking primitive and the dedicated connection it parks on.
- R3.01 · The processing list — the wait/active lists the woken worker fetches from.
- R3 · Reliable Queues — the chapter.
- E6 · The job lifecycle — the dedicated EchoMQ course: the marker / heartbeat coordination and the polyglot concurrency models.