jonnifyechomq
EchoMQ / The Protocol / Immutability & branded ids / The branded-id gate

Immutability · dive 02 · how

The branded-id gate

A job's id is a 14-byte branded Snowflake, and the wire checks it before a row exists.

A caller does not pick a job's id. It is a 14-byte branded id: a three-letter uppercase namespace, then eleven characters of Base62 Snowflake. For a job the namespace is JOB, and the @enqueue script checks it first — it reads the first three bytes and refuses anything that is not JOB, with the EMQKIND error class, before EXISTS, before HSET. Identity is enforced on the server, in Lua, as part of the same atomic step that writes the row. Decode a real id on the right, then read the gate that admits it.

The 14-byte branded id · pick one to decode

14 bytes = 3 namespace + 11 Base62(snowflake) JOBnamespace ·3 0KHTOWnGLuC Base62(snowflake) ·11 ts ·41 node ·10 seq ·12 snowflake = ts << 22 | node << 12 | seq

Identity is not a number the caller picks

A job's id is a 14-byte branded id: a three-letter uppercase namespace concatenated with an eleven-character Base62 encoding of a Snowflake. The layout is fixed — three plus eleven, always fourteen — and the Snowflake itself packs a 41-bit timestamp, a 10-bit node, and a 12-bit sequence (ts << 22 | node << 12 | seq) against a fixed epoch. For a job, the namespace is JOB.

The decoder above takes a real id apart: the namespace, the Snowflake, and the unpacked timestamp, node, and sequence — and the gate verdict. A JOB id is admitted; a USR id is the right shape but the wrong namespace, so the gate refuses it; a bare "job-42" is not a branded id at all. The branding is not a label — it is what the @enqueue script tests before the id may enter the job keyspace.

The gate, in two beats

Beat one — the named handle. EchoMQ.Jobs.enqueue/4 builds the declared keys and runs the @enqueue script through the connector, then maps the server's verdict back to a typed result — including the EMQKIND refusal the gate produces, which becomes {:error, :kind}.

# echo_mq — EchoMQ.Jobs
# The host builds the declared keys and runs @enqueue. The server's verdict
# is mapped: 1 enqueued, 0 a duplicate, and an EMQKIND error reply (the
# branded-id gate firing) becomes the typed {:error, :kind}.
def enqueue(conn, queue, job_id, payload) when is_binary(job_id) and is_binary(payload) do
  keys = [Keyspace.job_key(queue, job_id), Keyspace.queue_key(queue, "pending")]

  case Connector.eval(conn, @enqueue, keys, [job_id, payload]) do
    {:ok, 1} -> {:ok, :enqueued}
    {:ok, 0} -> {:ok, :duplicate}
    {:error, {:server, "EMQKIND" <> _}} -> {:error, :kind}
    other -> other
  end
end

Beat two — the gate, in the script body. The first statement of @enqueue is the namespace check. ARGV[1] is the 14-byte id; string.sub(ARGV[1], 1, 3) is its first three bytes. Not 'JOB' → redis.error_reply('EMQKIND …'), and the script stops before the idempotency check, before the row write, before the pending insert.

-- the @enqueue script — the branded-id gate is the first statement
-- ARGV[1] is the 14-byte branded id. Its first three bytes are the
-- namespace. A job id must be JOB-namespaced; anything else is refused at
-- the wire, before a row exists.
if string.sub(ARGV[1], 1, 3) ~= 'JOB' then
  return redis.error_reply('EMQKIND job id must be JOB-namespaced')
end
-- only a JOB-namespaced id reaches the idempotency check and the row write
if redis.call('EXISTS', KEYS[1]) == 1 then
  return 0
end
redis.call('HSET', KEYS[1], 'state', 'pending', 'attempts', '0', 'payload', ARGV[2])
redis.call('ZADD', KEYS[2], 0, ARGV[1])
return 1

A second gate runs earlier, host-side: EchoMQ.Keyspace.job_key/2 calls EchoData.BrandedId.valid?/1 and raises on an ill-formed id, so a malformed id never even reaches the key builder. The Lua gate is the wire's own check — the one that holds for every speaker in any runtime, because it lives in the shared script, not in one runtime's host code. Two gates, one for shape (host) and one for namespace (wire), and the row is written only past both.

A job id is a 14-byte branded Snowflake under JOB, and the gate is the first line of @enqueue — three bytes compared, the wrong namespace refused with EMQKIND before any write.

The gate, byte by byte

The gate is one comparison: string.sub(id, 1, 3) against 'JOB'. Pick a candidate id and read the exact bytes the script reads and the verdict it returns — admitted, or refused with EMQKIND before a row exists.

string.sub(id, 1, 3) ~= 'JOB' · pick a candidate

JOB sub(id,1,3) == 'JOB' admitted match → EXISTS → HSET → ZADD → return 1
the pattern (Redis Patterns Applied)

A branded key carries its type in its bytes, and the atomic script that touches it can check that type before it acts. Coordination teaches the atomic check-then-act move the gate is an instance of.

the implementation (echo_mq)

@enqueue's first statement is string.sub(ARGV[1], 1, 3) ~= 'JOB' → EMQKIND; the 14-byte id is JOB + a Base62 Snowflake; Keyspace.job_key/2 gates the shape again host-side via valid?/1.

Identity is checked in the shared script, not in one runtime's host code, so the gate holds for every speaker of the wire.

Recap — the gate and the id

A job id is a 14-byte branded Snowflake under JOB: EchoData.BrandedId fixes the layout (three namespace bytes + eleven Base62 bytes), and the @enqueue script's first line — string.sub(ARGV[1], 1, 3) ~= 'JOB' — refuses the wrong namespace with EMQKIND before any write. Keyspace.job_key/2 gates the shape host-side as well. The next dive reads the third guard — how a whole connection is fenced before it can touch the store.

Next in this module: The version fence — the {emq}:version reserved key the connector claims or verifies before its first command.

References

Where the branded-id contract and the scripting that enforces it are documented.

Sources

Related in this course