jonnifyechomq
EchoMQ / E3 · EchoMQ 2.0 / The owned keyspace

E3 · EchoMQ 2.0 · orientation · transparent hashtags by construction

The owned keyspace

EchoMQ 2.0 owns its keyspace. The emq: prefix replaces emq:, the per-queue hashtag is the core's job, and every queue is placed by construction.

EchoMQ 2.0 breaks from BullMQ on the first rung, and the most visible part of the break is the keyspace. emq.1 ships emq:{q}:<type> — the emq: prefix replacing emq:, the queue name brace-wrapped transparently by the core (a caller still writes Queue("orders")), and the braced base {emq}: reserved for the core's own cross-queue keys so the namespaces cannot collide. Every queue is slot-local by construction. The v1 line that interpolates the queue name verbatim under emq: freezes at 1.3.0. The rung is emq.1, drafted, taught from its spec.

v1 → v2 keyspace · queue payments

v1 LINE · frozen at 1.3.0 · verbatim, unplaced emq:payments:<type> no hashtag — nothing to place across threads

    emq.1 owns its keyspace

    EchoMQ 2.0 drops BullMQ wire compatibility on the first rung, and the most visible part of that break is the keyspace. The v1 line — echo/apps/echomq v1.3.0, the line the fork freezes — speaks the emq: prefix and interpolates the queue name verbatim: the shipped EchoMQ.Keys.base/1 returns "#{prefix}:#{name}" (keys.ex:97-98), key_prefix/1 returns "#{prefix}:#{name}:" (keys.ex:85-86) — no wrapping, no validation, no placement. A queue named payments produces emq:payments:wait, emq:payments:active, and the rest: flat strings with nothing for a thread-per-shard datastore to balance on.

    emq.1 ships the owned keyspace. Every per-queue key becomes emq:{q}:<type> — the emq: prefix replacing emq:, the queue name brace-wrapped transparently by the core (a caller still writes Queue("orders") and the core places emq:{orders}:*). The wrap is idempotent (an already-braced caller name is not double-wrapped) and validated (no colon inside a hashtag; emq is rejected as a queue name). The braced base {emq}: is reserved for the core's own cross-queue keys — so the core's namespace and any queue's cannot collide (INV3). Every queue is slot-local by construction.

    v1 line (1.3.0)
    The frozen emq: keyspace: EchoMQ.Keys interpolating the queue name verbatim (keys.ex:85-86), EchoMQ.Version = bullmq:5.65.1. Flat, unplaced.
    emq:{q}:<type> (emq.1 ships)
    The owned v2 keyspace: emq: prefix, the per-queue {q} hashtag applied transparently (idempotent, validated), every queue slot-local.
    {emq}: (reserved)
    The braced base reserved for the core's own cross-queue keys; emq rejected as a queue name so the namespaces cannot collide (INV3).

    The rung is drafted. lib/echomq/ext/ and the v2 script set do not exist yet, so every emq:{q}:* surface here is taught from specs/emq/emq.1.md, written "emq.1 ships …" — never asserted as shipped. The v1 anchors (EchoMQ.Keys, bullmq:5.65.1) are real and present.

    The v1 → v2 key comparison, on the real grounding

    Take the fixed queue payments. The v1 line, today, produces the emq: families — the verbatim-interpolated builders in EchoMQ.Keys: the LIST keys :wait / :active / :paused, the ZSET keys :delayed / :prioritized / :completed / :failed / :waiting-children, the SET :stalled, the STREAM :events, the HASH :meta, and the per-job emq:payments:<jobId>:lock via EchoMQ.Keys.lock/2. These are flat: no hashtag, nothing to place across threads. This is the line the fork freezes at 1.3.0.

    emq.1 ships the v2 form of each. emq:payments:wait becomes emq:{payments}:wait, emq:payments:active becomes emq:{payments}:active, … emq:payments:<jobId>:lock becomes emq:{payments}:<jobId>:lock. The emq: prefix replaces emq:; the queue name carries its {…} hashtag, applied transparently by the core; the field set is otherwise the same. The hero figure shows each line's key set side by side — toggle it and watch the prefix flip and the hashtag appear.

    # queue "payments" — v1 line (frozen at 1.3.0) → emq.1 ships (2.0).
    # v1: EchoMQ.Keys interpolates the name verbatim under emq: (keys.ex:85-86) — flat, unplaced
      emq:payments:wait   emq:payments:active   emq:payments:meta
      emq:payments:<jobId>:lock   # EchoMQ.Keys.lock/2 — no hashtag
    # emq.1 ships: emq: prefix, the {q} hashtag applied transparently by the core
      emq:{payments}:wait   emq:{payments}:active   emq:{payments}:meta
      emq:{payments}:<jobId>:lock   # one hashtag {payments} — slot-local by construction
      {emq}:version  {emq}:nodes   # the core's own keys, the reserved {emq}: base

    emq.1 ships the break: emq: for emq:, and the queue name brace-wrapped so every v2 key is placed by construction.

    Two slots, by construction

    emq.1 ships the transparent hashtag so a queue's keys are placed precisely. A datastore that locks on the hashtag routes a key by the substring inside the first {…} — the hashtag. So every v2 key of the payments queue — emq:{payments}:wait, emq:{payments}:active, emq:{payments}:<jobId>:lock — carries the same tag, payments, and lands in the same slot. That is what lets a v2 Lua script touch several of a queue's keys in one atomic multi-key transition — the R2 atomicity floor, now placement-native.

    The core's own cross-queue keys live under the reserved {emq}: base — {emq}:version, {emq}:nodes — and group into a second, distinct slot: the queue's and the core's. They cannot collide, because emq is rejected as a queue name (INV3). A property test over arbitrary queue names checks the rule holds: wrap°wrap = wrap, and every per-queue key shares exactly the queue's one hashtag (emq.1-D1). The figure below extracts the {…} tag from each key and groups by tag — the queue's keys to one slot, the core's to the other.

    Extract the hashtag · group into two slots

    emq:{payments}:wait emq:{payments}:active emq:{payments}:<jobId>:lock {emq}:version tag {payments} the queue's slot tag {emq} the core's reserved slot
    the pattern · ← redis-patterns R2 (atomicity)

    A change is one indivisible move, foreign to other state — the atomicity floor a queue's correctness rests on, and the foundations (R0) that make the keyspace and the scripts the protocol.

    the implementation · ⇄ emq.1 (drafted · the 2.0 break)

    emq.1 ships the owned keyspace: emq:{q}:* with the brace applied by the core, every queue slot-local, and the core's own keys alone under the reserved {emq}: (INV3) — so a single hashtag per transition keeps it single-slot and atomic.

    Same {tag} ⇒ same slot. emq.1 ships a keyspace placed by construction: one queue per slot, the core's own keys alone in their reserved slot — the foundation for locking on the hashtag instead of the whole store.

    The 2.0 fork — EchoMQ leaves the BullMQ wire

    The verbatim emq:payments:* keyspace is the v1 line's, frozen at 1.3.0 (E2 · the core teaches it). emq.1 ships the break: emq: replaces emq:, the per-queue {q} hashtag is applied transparently by the core, and {emq}: is reserved for the core's own keys — so DragonflyDB can lock on the hashtag precisely instead of the whole store.

    Recap — emq.1 owns its keyspace

    The v1 line interpolates the queue name verbatim under emq: (EchoMQ.Keys, keys.ex:85-86) — flat, unplaced, frozen at 1.3.0. emq.1 ships emq:{q}:<type>: the emq: prefix replacing emq:, the per-queue {q} hashtag applied transparently (idempotent, validated), the braced base {emq}: reserved for the core's own cross-queue keys, and emq rejected as a queue name so the namespaces cannot collide (INV3). Every queue is slot-local by construction, so a hashtag-locking datastore places one queue per thread and distinct queues across cores.

    Next in E3: The versioned break — how emq.1 ships echomq:2.0.0, the two-way boot fence that refuses cross-version contact, and the explicit v1→v2 migration path, dispatched the proven way EchoMQ.Scripts.execute_raw/4 dispatches today.

    References

    Where the owned keyspace is grounded: the BullMQ wire the v1 line speaks and the fork leaves, the Dragonfly hashtag-locking the transparent brace targets, the cluster hashtag rule, and the chapter's own pages.

    Sources

    Related in this course