Dive C4.08.1

Keys & rows

The branded id is the key. The row is what the key addresses. Build one from the other.

A persisted entity needs a key and a place to live. In EchoMQ the key is built from a branded id: EchoMQ.Keyspace.queueKey composes a per-queue key as emq:{q}:<type>, and jobKey appends a valid branded id to the queue's job: family — emq:{q}:job:<id>. The braced part is a Redis Cluster hashtag: every key of one queue shares it, so they route to one slot, computed by CRC16 of the hashtag modulo 16384. The row that key points at is the persisted entity — a record keyed by the id. This dive composes the key, gates the id, hashes the slot, and names the row, all in valid F# verified in dotnet fsi.

Build a key Module hub

C4.08.1 · part 1

The key is composed from the id

A key in this keyspace has three fixed parts and one variable one: the literal emq, the braced queue name, a type segment, and — for a job — the branded id. queueKey builds the per-queue prefix; jobKey appends the id to the job: family. The id is checked before it is used: an ill-formed id raises rather than minting a key the store would then have to reject.

// echo_fs/EchoMQ/Keyspace.fs — quoted verbatim
module Keyspace =

    /// A per-queue key: `emq:{q}:<type>`.
    let queueKey (queue: string) (keyType: string) : string =
        "emq:{" + queue + "}:" + keyType

    /// A job key: the queue's `job:` family plus a valid branded id.
    let jobKey (queue: string) (branded: string) : string =
        if BrandedId.isValid branded then
            queueKey queue "job:" + branded
        else
            invalidArg (nameof branded) "jobKey requires a valid branded id"

Run in dotnet fsi: Keyspace.jobKey "email" "JOB0Dqhu1EacDY" returns "emq:{email}:job:JOB0Dqhu1EacDY". The branded id is the key; the rest is grammar.

queueKey: queue -> keyType -> string — the per-queue prefix, e.g. emq:{email}:pending.
jobKey: queue -> branded -> string — the row key for one job; raises on an invalid id.
hashtag: the substring inside the first {...} — the part Redis Cluster hashes for routing.
slot: key -> int — CRC16 of the hashtag, modulo 16384; the cluster slot the key lives in.

C4.08.1 · part 2

The id is gated before it is a key

A branded id is fourteen characters: three uppercase letters of namespace and eleven Base62 characters of snowflake. BrandedId.isValid checks exactly that shape. A bare "JOB" has a valid namespace but is not a valid id — it is three chars, not fourteen. jobKey calls this check first, so a malformed id never becomes a key. The store therefore only ever holds keys that could exist.

// echo_fs/EchoMQ/BrandedId.fs — quoted verbatim
/// 3 uppercase letters then 11 Base62 chars. A bare "JOB" has a valid
/// namespace but is not a valid id (the gate the key builder enforces).
let isValid (id: string) : bool =
    not (isNull id)
    && id.Length = 14
    && Seq.forall isUpper (id.Substring(0, 3))
    && Seq.forall isBase62 (id.Substring(3))

A valid id

"JOB0Dqhu1EacDY" — 14 chars, 3 upper + 11 Base62 — composes to a key.

An invalid id

"job" — wrong case, wrong length — makes jobKey raise. No key is built.

Build the key, hash the slot

Pick a queue and a branded id. The builder composes emq:{q}:job:<id> and computes its cluster slot — CRC16-XMODEM of the hashtag, modulo 16384. The queue 123456789 is the published test vector: its hashtag hashes to slot 12739. Pick the invalid id to watch the gate refuse to compose a key. The readout names each part and shows the verified FSI echo.

The keyspace builder · pick a queue and an id

Every key of one queue carries the same hashtag, so every key of one queue lands on one slot. The id is the variable; the slot is fixed per queue.

C4.08.1 · part 3

The row is what the key points at

The key locates the entity; the row is the entity. In the F# port the rows are an immutable Map<string, Job> keyed by branded id — Store.QueueState.Rows. A Job carries its id, its lifecycle state, the attempt count (which doubles as the lease's fencing token), the payload, and the optional fair-lane group, progress note, and last error.

// echo_fs/EchoMQ/Job.fs — the row, quoted verbatim
type Job =
    { Id: string
      State: JobState
      Attempts: int        // doubles as the lease's fencing token
      Payload: string
      Group: string option
      Progress: string option
      LastError: string option }

// echo_fs/EchoMQ/Store.fs — the rows are keyed by branded id
type QueueState =
    { Rows: Map<string, Job>
      Pending: Set<string>
      // schedule / active / dead — the rest of the four-set lifecycle
      // ... }

The F# cost model: Map is a balanced tree, so a lookup or insert by id is O(log n) and an insert shares most of the previous tree's nodes — the persistent-map structural sharing C4.05 and C4.06 build up. On the BEAM the Elixir bus reaches the same shape through its CHAMP; on .NET an immutable Map is the direct equivalent.

EchoMQ in F#

Status — echo_fs is an in-memory model. The EchoMQ protocol — the Lua transitions executing on a Valkey wire — is not implemented (there is no Lua and no Valkey client in the code); that is the coming-soon transport (EchoMQ.Connector). This block shows how the model uses the structure, not a running bus.

The bus uses every piece of this dive. Its row keys are composed from branded ids; its rows are the persisted entities; its hashtag pins a queue to one slot. Grounded in the real echo_fs port.

echo_fs · Keyspace.queueKey / jobKey / slot, Store.Rows

A job's row key is Keyspace.jobKey queue id = emq:{q}:job:<id> — the branded id composed onto the queue's job: family. The row it addresses is one entry in Store.QueueState.Rows : Map<string, Job>. The queue's hashtag — Keyspace.hashtag — pins every one of its keys to a single cluster slot via Keyspace.slot, CRC16 of the hashtag modulo 16384, with the published vector slot "123456789" = 12739.

The in-memory Store holds these rows today; the Valkey transport EchoMQ.Connector lands next — the same keys, the same rows, behind the wire.

Where this leads

The key is built and the row is named. The next dive asks what is actually stored in the row, and why the key form — a Base62 string — survives the round trip across SQLite, PostgreSQL, and Redis as a sortable key. That is the F4.08 parity point.

↑ C4.08 · Persistence C4.08.2 · Serialization → C4.08.3 · The store port

References

Sources

Microsoft — F# documentation — records and the Map the rows are held in.
FSharp.Core API reference — Map and Seq.forall, used by the row map and the id gate.
Microsoft — .NET documentation — System.Text.Encoding and the bit operators the CRC16 slot uses.
The F# Software Foundation — F# on .NET, the home of the echo_fs port.

Related in this course

Elixir F4.08.1 — Branded ids as keys — the parity dive on the BEAM.
C4.08 — Branded ids & persistence — the module hub.
C4.08.2 — Serialization — what the row stores.
C4 — Algorithms & Data Structures — the chapter.