jonnifyechomq
EchoMQ / The Protocol / The owned keyspace / The braced grammar

The owned keyspace · dive 01 · the grammar

The braced grammar

One shape for every key: emq, the queue in braces, the type. Two productions, one builder.

Every key EchoMQ writes has the same shape — emq:{q}:<type>. This is the L1 grammar, and it is total: no key is written that EchoMQ.Keyspace did not compose. This dive reads its two productions on the real code — queue_key/2 for the per-queue key and job_key/2 for the job row — and shows why one queue's keys all carry the same {q} hashtag.

The shape queue_key/2 job_key/2 Pattern → implementation References

A per-queue key · pick a segment

emq: {orders} :pending literal hashtag (the queue) type queue_key("orders", "pending")

The shape, and why it is total

A protocol's data layer is its agreement about where things live. EchoMQ's agreement is one string shape: emq:{q}:<type> — the literal emq:, the queue name wrapped in braces, a colon, then the type. The type is the role of the key within the queue: pending (the ready set), active (the claimed set), schedule (the delayed set), job:<id> (a record row).

What makes it a protocol and not a habit is that the grammar is total: there is no path by which the system writes a key that EchoMQ.Keyspace did not build. The module has exactly two productions for queue state — queue_key/2 and job_key/2 — and the job row is itself a per-queue key with a validated id appended. So one queue named orders owns a whole family of keys that all share the {orders} hashtag.

emq:{orders}:pending
the ready set — ids waiting to be claimed.
emq:{orders}:active
the claimed set — ids a worker holds.
emq:{orders}:schedule
the delayed set — ids parked until a due time.
emq:{orders}:job:<id>
the record row — one job's hash.

Every key in that family carries {orders}, so every key routes to one slot — the precondition for the atomic multi-key Lua of The Lua layer.

queue_key/2 — the per-queue production

queue_key/2 composes the per-queue key as iodata, then flattens it to a binary in one pass. The guard fixes both arguments as binaries, so a non-string queue or type fails at the boundary rather than producing a malformed key.

# echo_mq — EchoMQ.Keyspace
# Per-queue keys are emq:{q}:<type>. The braces are not decoration —
# they are the cluster hashtag, and they pin every key of one queue to
# one slot. Built as iodata, then flattened once.
def queue_key(queue, type) when is_binary(queue) and is_binary(type),
  do: IO.iodata_to_binary(["emq:{", queue, "}:", type])

The iodata list ["emq:{", queue, "}:", type] is the grammar made literal: the brace opens immediately after emq: and closes immediately before the type colon, so the queue name is always the hashtag and nothing else is. There is no string interpolation that a caller could subvert — the structure is in the list, not in a format string.

job_key/2 — the job row, gated

The job row is a per-queue key whose type is job: with a validated branded id appended. It is the one production that refuses its input: an id that is not a valid branded id is not turned into a key — the function raises.

# echo_mq — EchoMQ.Keyspace
# The job row is a per-queue key whose type is "job:" plus the branded id.
# The id is GATED first: the keyspace refuses to address an unbranded row,
# so a malformed id never becomes a key.
def job_key(queue, branded) when is_binary(branded) do
  if EchoData.BrandedId.valid?(branded) do
    queue_key(queue, "job:") <> branded
  else
    raise ArgumentError, "job_key requires a valid branded id"
  end
end

So job_key("orders", "JOB0Nb1VTbfnu4") yields emq:{orders}:job:JOB0Nb1VTbfnu4. The branded id is the long part of the key by design — 14 bytes, a 3-byte namespace plus an 11-byte Base62 payload — and valid?/1 is the first place the wire enforces it. The id is gated a second time inside the Lua, on its kind prefix; that is Immutability & branded ids. Here the point is the grammar: the row's address is always a per-queue key, so it co-locates with the queue's sets.

Build a key — read off the grammar

The grammar is mechanical: give a queue name and a type and the key is determined. Type a queue and pick a type below. The page composes the key exactly as queue_key/2 does — and, for the job: type, appends a sample branded id the way job_key/2 does — then reads back the hashtag (always the queue name) so you can see the brace is the routing key.

queue_key / job_key · compose a key

A queue name and a type determine the key — and the key always carries the queue as its hashtag. That is the whole data-layer agreement: one shape, one builder, one routing key per queue.

Pattern → implementation

the pattern (Redis Patterns Applied)

A key naming convention becomes a contract when every writer agrees on the exact string shape — the prefix, the separators, the field positions — and on the hashtag that co-locates a group. Redis Patterns Applied R2.05 teaches it.

the implementation (echo_mq)

EchoMQ.Keyspace.queue_key/2 and job_key/2 make emq:{q}:<type> the only way a key is built, so the convention is total — the grammar is the protocol, not a style guide, and the brace makes every queue single-slot.

The grammar is the data-layer half of the protocol. Build every key through one module to one braced shape, and "where things live" stops being a convention and becomes the wire.

Recap — one shape, two productions

The braced grammar is emq:{q}:<type>. queue_key/2 builds the per-queue key as iodata; job_key/2 builds the job row from a per-queue key plus a validated branded id, raising on an invalid one. The grammar is total — no key escapes the builder — and every key of one queue shares the {q} hashtag, so they co-locate on one slot. The next dive reads what that hashtag computes: the cluster slot.

Next in this module: The hashtag & the slot — the brace as the cluster hashtag, and slot/1 = CRC16-XMODEM % 16384.

References

Where the grammar and the hashtag convention are documented, and where the record it addresses is read.

Sources

Related in this course