E3 · EchoMQ 2.0 · orientation · transparent hashtags by construction
The owned keyspace
EchoMQ 2.0 owns its keyspace. The emq: prefix replaces
emq:, the per-queue hashtag is the core's job, and every queue is placed by
construction.
EchoMQ 2.0 breaks from BullMQ on the first rung, and the most visible part
of the break is the keyspace. emq.1 ships emq:{q}:<type> — the
emq: prefix replacing emq:, the queue name brace-wrapped
transparently by the core (a caller still writes Queue("orders")),
and the braced base {emq}: reserved for the core's own cross-queue keys so the
namespaces cannot collide. Every queue is slot-local by construction. The v1 line that interpolates the queue
name verbatim under emq: freezes at 1.3.0. The rung is
emq.1, drafted, taught from its spec.
Movement II · Extension · tracks emq.1 · drafted (the 2.0 break) ← redis-patterns R0 ← redis-patterns R2
v1 → v2 keyspace · queue payments
emq.1 owns its keyspace
EchoMQ 2.0 drops BullMQ wire compatibility on the first rung, and the most visible part of
that break is the keyspace. The v1 line — echo/apps/echomq v1.3.0,
the line the fork freezes — speaks the emq: prefix and interpolates the queue
name verbatim: the shipped EchoMQ.Keys.base/1 returns
"#{prefix}:#{name}" (keys.ex:97-98),
key_prefix/1 returns "#{prefix}:#{name}:"
(keys.ex:85-86) — no wrapping, no validation, no placement. A queue named
payments produces emq:payments:wait,
emq:payments:active, and the rest: flat strings with nothing for a thread-per-shard
datastore to balance on.
emq.1 ships the owned keyspace. Every per-queue key becomes
emq:{q}:<type> — the emq: prefix replacing
emq:, the queue name brace-wrapped transparently by the core (a
caller still writes Queue("orders") and the core places
emq:{orders}:*). The wrap is idempotent (an already-braced caller
name is not double-wrapped) and validated (no colon inside a hashtag;
emq is rejected as a queue name). The braced base {emq}: is
reserved for the core's own cross-queue keys — so the core's namespace and any queue's
cannot collide (INV3). Every queue is slot-local by construction.
- v1 line (1.3.0)
- The frozen
emq:keyspace:EchoMQ.Keysinterpolating the queue name verbatim (keys.ex:85-86),EchoMQ.Version=bullmq:5.65.1. Flat, unplaced. - emq:{q}:<type> (emq.1 ships)
- The owned v2 keyspace:
emq:prefix, the per-queue{q}hashtag applied transparently (idempotent, validated), every queue slot-local. - {emq}: (reserved)
- The braced base reserved for the core's own cross-queue keys;
emqrejected as a queue name so the namespaces cannot collide (INV3).
The rung is drafted. lib/echomq/ext/ and the v2 script
set do not exist yet, so every emq:{q}:* surface here is taught from
specs/emq/emq.1.md, written "emq.1 ships …" — never asserted as
shipped. The v1 anchors (EchoMQ.Keys, bullmq:5.65.1) are real
and present.
The v1 → v2 key comparison, on the real grounding
Take the fixed queue payments. The v1 line, today, produces the
emq: families — the verbatim-interpolated builders in
EchoMQ.Keys: the LIST keys :wait / :active /
:paused, the ZSET keys :delayed / :prioritized /
:completed / :failed / :waiting-children,
the SET :stalled, the STREAM :events, the HASH
:meta, and the per-job emq:payments:<jobId>:lock via
EchoMQ.Keys.lock/2. These are flat: no hashtag, nothing to place across threads. This
is the line the fork freezes at 1.3.0.
emq.1 ships the v2 form of each. emq:payments:wait becomes
emq:{payments}:wait, emq:payments:active becomes
emq:{payments}:active, … emq:payments:<jobId>:lock
becomes emq:{payments}:<jobId>:lock. The emq: prefix
replaces emq:; the queue name carries its {…}
hashtag, applied transparently by the core; the field set is otherwise the same. The hero figure shows each
line's key set side by side — toggle it and watch the prefix flip and the hashtag appear.
# queue "payments" — v1 line (frozen at 1.3.0) → emq.1 ships (2.0). # v1: EchoMQ.Keys interpolates the name verbatim under emq: (keys.ex:85-86) — flat, unplaced emq:payments:wait emq:payments:active emq:payments:meta emq:payments:<jobId>:lock # EchoMQ.Keys.lock/2 — no hashtag # emq.1 ships: emq: prefix, the {q} hashtag applied transparently by the core emq:{payments}:wait emq:{payments}:active emq:{payments}:meta emq:{payments}:<jobId>:lock # one hashtag {payments} — slot-local by construction {emq}:version {emq}:nodes # the core's own keys, the reserved {emq}: base
emq.1 ships the break: emq: for emq:, and the
queue name brace-wrapped so every v2 key is placed by construction.
Two slots, by construction
emq.1 ships the transparent hashtag so a queue's keys are placed precisely. A datastore that locks on the
hashtag routes a key by the substring inside the first {…} — the hashtag.
So every v2 key of the payments queue — emq:{payments}:wait,
emq:{payments}:active, emq:{payments}:<jobId>:lock —
carries the same tag, payments, and lands in the same slot. That is
what lets a v2 Lua script touch several of a queue's keys in one atomic multi-key transition — the R2
atomicity floor, now placement-native.
The core's own cross-queue keys live under the reserved {emq}: base —
{emq}:version, {emq}:nodes — and group into a
second, distinct slot: the queue's and the core's. They cannot collide, because
emq is rejected as a queue name (INV3). A property test over arbitrary queue names
checks the rule holds: wrap°wrap = wrap, and every per-queue key shares exactly the
queue's one hashtag (emq.1-D1). The figure below extracts the {…} tag from each
key and groups by tag — the queue's keys to one slot, the core's to the other.
Extract the hashtag · group into two slots
A change is one indivisible move, foreign to other state — the atomicity floor a queue's correctness rests on, and the foundations (R0) that make the keyspace and the scripts the protocol.
emq.1 ships the owned keyspace: emq:{q}:* with the brace applied by the core, every
queue slot-local, and the core's own keys alone under the reserved {emq}: (INV3) —
so a single hashtag per transition keeps it single-slot and atomic.
Same {tag} ⇒ same slot. emq.1 ships a keyspace placed by construction:
one queue per slot, the core's own keys alone in their reserved slot — the foundation for locking on the
hashtag instead of the whole store.
The 2.0 fork — EchoMQ leaves the BullMQ wire
The verbatim emq:payments:* keyspace is the v1 line's, frozen at
1.3.0 (E2 · the core teaches it). emq.1 ships the
break: emq: replaces emq:, the per-queue
{q} hashtag is applied transparently by the core, and {emq}:
is reserved for the core's own keys — so DragonflyDB can lock on the hashtag precisely instead of the
whole store.
Recap — emq.1 owns its keyspace
The v1 line interpolates the queue name verbatim under emq:
(EchoMQ.Keys, keys.ex:85-86) — flat, unplaced, frozen
at 1.3.0. emq.1 ships emq:{q}:<type>: the
emq: prefix replacing emq:, the per-queue
{q} hashtag applied transparently (idempotent, validated), the braced base
{emq}: reserved for the core's own cross-queue keys, and emq
rejected as a queue name so the namespaces cannot collide (INV3). Every queue is slot-local by construction,
so a hashtag-locking datastore places one queue per thread and distinct queues across cores.
Next in E3: The versioned break — how emq.1 ships
echomq:2.0.0, the two-way boot fence that refuses cross-version contact, and the
explicit v1→v2 migration path, dispatched the proven way EchoMQ.Scripts.execute_raw/4
dispatches today.
References
Where the owned keyspace is grounded: the BullMQ wire the v1 line speaks and the fork leaves, the Dragonfly hashtag-locking the transparent brace targets, the cluster hashtag rule, and the chapter's own pages.
Sources
- BullMQ — Documentation — the reference implementation whose
emq:key layout the v1 line speaks and the 2.0 fork leaves behind. - DragonflyDB — Server flags —
--lock_on_hashtags("locks are done at the{hashtag}level"), the mechanism the transparent{q}hashtag targets. - DragonflyDB — BullMQ on Dragonfly — the multithreading ceiling of an unplaced keyspace that the owned keyspace removes.
- Redis — Cluster specification — the hash-tag rule (
{…}decides the slot) the transparent brace relies on.
Related in this course
- E3 · EchoMQ 2.0 — the chapter this dive opens.
- E2 · The core — the v1 line the fork forks from.
- redis-patterns R2 · Coordination — atomicity, the pattern this dive applies.
- redis-patterns R0 · Overview — foundations: why the keyspace and the scripts are the protocol.