jonnifyfsharp

Dive C4.09.1

The branded CHAMP

A queue needs a place to keep its jobs. The place is a persistent map, and the key is the id.

The registry behind a queue holds one row per job, found by the job's id. This dive builds it from the chapter's parts: the persistent map of C4.05–C4.06, keyed by the branded Snowflake of C4.07. Because the map is persistent — every write returns a new version that shares structure with the old one — a snapshot is free and an old root is never mutated, which is what lets reads run without a lock. The compressed form of that map is the CHAMP: the production registry keyed by branded JOB ids. In the echo_fs port the registry is EchoMQ.Store.Rows, a Map<string, Job> today — a plain persistent map, not yet a CHAMP, by design. The branded CHAMP rides the same Store surface when it lands.

C4.09.1 · part 1

The key is the id

A job's identity is a branded id: a three-letter namespace plus an eleven-character Base62 Snowflake (C4.07). EchoMQ jobs live under the JOB namespace. That string is the registry's key. There is no separate row number and no second index — the id you already hold is the whole address, so a lookup is one map descent and an admission is one key check.

The row itself is a record: the id, the lifecycle state, the attempt count (which also serves as the lease's fencing token), the payload, and a few options. This is the echo_fs Job type, the value the map holds against each key.

// echo_fs/EchoMQ/Job.fs — the value the registry holds
type Job =
    { Id: string                  // a JOB-namespaced branded id — the map key
      State: JobState
      Attempts: int               // doubles as the lease's fencing token
      Payload: string
      Group: string option
      Progress: string option
      LastError: string option }
branded id
a 3-letter namespace + an 11-char Base62 snowflake (14 chars). JOB is the queue's namespace; the string is the key.
row
the Job record stored against the key — its state, attempts, payload, and options.
registry
a persistent map from id to row. One key, one row; a lookup is a single map descent.
CHAMP
the compressed persistent map (C4.06) — the production form of the registry, keyed by branded ids.
C4.09.1 · part 2

A write returns a new map; the old one stands

The registry is immutable. Adding a row does not change the map in place — it returns a new map that shares almost all of its structure with the previous one. The old map is still valid and still holds its old rows. That is the property the chapter has been building toward: because no root is ever mutated, a reader walking last moment's map cannot be disturbed by a writer building the next one.

This is plain in dotnet fsi. Add a row to a map and the original's count is unchanged; the new binding holds the extra row. Two versions coexist, the cost of the second being one root-to-leaf path, not a copy of the whole structure.

type JobState = Pending | Active
type Job = { Id: string; State: JobState; Attempts: int; Payload: string }

let rows : Map<string, Job> = Map.empty
// val rows: Map<string,Job> = map []

let rows1 = rows.Add("JOB0Nc1aZxKq9P",
                     { Id = "JOB0Nc1aZxKq9P"; State = Pending; Attempts = 0; Payload = "build #42" })
rows1.Count   // val it: int = 1
rows.Count    // val it: int = 0  — the original map is untouched
The original rows still reports 0 after the add. Persistence is not a copy: rows1 shares its structure with rows and adds one path. This is the structural sharing of C4.05–C4.06, here on the registry.

Write a row, keep the old map

Add a row to the registry. Each accepted write returns a new map version that holds one more row and shares the rest with the version before it; the readout names the new count and confirms the prior version is unchanged. A second write of a key already present is refused — a map cannot hold two rows under one key — so the count holds and the verdict is a duplicate. Every step is the real Map operation over a fixed set of ids.

Persistent Map<string, Job> · add a row

previous version 0 rows Map.empty .Add new version 0 rows Map.empty

      
Add a row to write a new map version. The previous version keeps its rows unchanged — the new one shares them and adds one.

Every write is a new map that keeps the last one. The registry is a chain of immutable versions, each sharing structure with its predecessor — so a snapshot costs one path, and an old reader is never disturbed.

C4.09.1 · part 3

The CHAMP is the production form

F#'s built-in Map is a balanced tree, and it has all the properties the registry needs: persistent, immutable, O(log n) lookup and insert. The CHAMP of C4.06 keeps the same persistence and improves the constants — a wider branching factor and a compressed node layout that packs data and sub-nodes separately, so iteration is cache-friendlier and equality is cheaper. For a registry keyed by branded ids, the CHAMP is the form to reach for at scale.

The point for this module is that the choice of map does not change the surface. Whether the rows live in a balanced Map or a branded CHAMP, the registry is still id → row, still persistent, still owned by one writer. The next two dives build that writer; the map underneath it can be swapped without touching them.

C4.05 + C4.06 + C4.07

A persistent map, a compressed CHAMP layout, and a branded id key.

C4.09.1 · the registry

One persistent map keyed by branded JOB ids — the store of a queue, ready for a single writer.

EchoMQ in F#

Status — echo_fs is an in-memory model. The EchoMQ protocol — the Lua transitions executing on a Valkey wire — is not implemented (there is no Lua and no Valkey client in the code); that is the coming-soon transport (EchoMQ.Connector). This block shows how the model uses the structure, not a running bus.

In the F# bus the registry is EchoMQ.Store.Rows — the rows field of the queue's state, keyed by branded id. The store's own comment names the property the chapter has been building toward.

runs today echo_fs/EchoMQ/Store.fs

The queue's state holds Rows: Map<string, Job> — the branded-keyed registry — beside an ordered Pending: Set<string>. The module comment records the design: "Rows are a Map keyed by branded id; the pending set is an ordered Set of ids. Because branded ids are time-ordered and Base62 sorts as the integers do, the id order IS the mint order — the 'no second index' property of the Elixir bus, here for free from Set's ordering." Admission is idempotent: Store.enqueue refuses an id already present and otherwise writes one row and adds the id to pending.

coming soon The registry is a balanced Map today, not the compressed branded CHAMP — the store's comment states as much, calling this model "the skeleton's source of truth" with "the Valkey-backed store … coming soon". The CHAMP is the production form of the same surface; it slots in behind Store.Rows without changing the broker that owns it.

Where this leads

The registry exists and shares structure on every write. What it still lacks is a writer — the one thing allowed to add and remove rows, so the versions form a single line rather than a tangle. The next dive builds that writer: EchoMQ.Broker, an F# MailboxProcessor that owns the store and threads a new version on every message.

References

Sources

Related in this course