Dive C4.08.2

Serialization & the sortable key

The payload is an opaque string. The key is a Base62 id. And the key form is what makes the store free of a sort column.

This is the F4.08 parity point, re-derived in F#. The same branded id serves three stores: a 64-bit bigint primary key in SQLite and PostgreSQL, and a namespaced string key in Redis and Valkey. The decisive property is that the key sorts by mint time. The Base62 alphabet runs in ascending code-point order, and the snowflake carries the timestamp in its high bits, so a fixed-width string sorts lexicographically exactly as the integer sorts: byte order is mint order. So a Set<string> of ids, or a Map<string, _> keyed on them, is already in mint order — no created_at column, no second index. This dive shows the row's shape, the alphabet that earns the order, and the order made visible, all verified in dotnet fsi.

See the order Module hub

C4.08.2 · part 1

What is actually stored

A row reduces to a flat, store-neutral record: the id (the key), a lifecycle state, the attempt count, and the payload. The payload is an opaque string — JSON here, but the store does not care; it carries the bytes and the id indexes them. Job.create produces a fresh Pending row with zero attempts.

// echo_fs/EchoMQ/Job.fs — a fresh pending row, quoted verbatim
let create (id: string) (payload: string) : Job =
    { Id = id
      State = JobState.Pending
      Attempts = 0
      Payload = payload
      Group = None; Progress = None; LastError = None }

// the row, flattened for any store — verified in dotnet fsi:
let row =
    {| id = "JOB0Dqhu1EacDY"; state = "pending"
       attempts = 0; payload = "{\"to\":\"ada@x.io\"}" |}
// val row : {| attempts: int; id: string; payload: string; state: string |}

The key is the id; the payload is bytes. SQLite and PostgreSQL store the id as a bigint primary key and the payload in a text or JSON column; Redis stores the row under the string key emq:{q}:job:<id>. The shape is the same; only the column type differs.

C4.08.2 · part 2

Why byte order is mint order

Two facts combine. First, the Base62 alphabet is 0-9 A-Z a-z — in ascending code-point order. So a fixed-width big-endian encoding sorts lexicographically exactly as the integers sort. Second, the snowflake puts the timestamp in its high bits, so a later mint is always a larger integer. Composed: the id string sorts by mint time. The Elixir bus calls this "byte order is mint order"; the F# port's Store comment names it as "the id order IS the mint order — the 'no second index' property."

// echo_fs/EchoMQ/Base62.fs — ascending code-point order, quoted verbatim
let Alphabet = "0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz"

// two ids minted 1 ms apart — verified in dotnet fsi:
"JOB0Dqhu1EacDY" < "JOB0Dqhu1EsDLc"   // val it : bool = true  (string order)
187490421964828672L < 187490421964832768L   // val it : bool = true  (integer order)

Without JavaScript: the same id, three stores
store	key form	sorted by
SQLite	INTEGER primary key (the 64-bit snowflake)	the integer — already mint order
PostgreSQL	bigint primary key (the 64-bit snowflake)	the integer — already mint order
Redis / Valkey	string key emq:{q}:job:<id>	the Base62 bytes — already mint order

The order made visible

Three jobs minted at different seconds, listed in whatever order they arrived. Put them in a Set<string>, and Set.toList hands them back in lexicographic order — which, for branded ids, is mint order. Press shuffle to scramble the input; the Set view always settles into the same time order. No created_at column is read; the order is in the key.

Insertion order vs Set order · three branded ids

The store never reads a timestamp to order these rows. The order is the key, and the key is the id. That is the whole F4.08 parity point.

EchoMQ in F#

Status — echo_fs is an in-memory model. The EchoMQ protocol — the Lua transitions executing on a Valkey wire — is not implemented (there is no Lua and no Valkey client in the code); that is the coming-soon transport (EchoMQ.Connector). This block shows how the model uses the structure, not a running bus.

The bus stores exactly this way: the payload is a string, the branded id is the sortable key, and the row is a flat record. Grounded in the real echo_fs port.

echo_fs · Base62.Alphabet, Store.Pending, Store.Rows

The pending set is Store.QueueState.Pending : Set<string> — branded ids only. Because Base62.Alphabet is ascending and the snowflake is timestamp-high, the Set is already in mint order, so Store.claim can pop the oldest with Set.minElement and Store.browse can read newest-first with List.rev — neither needs a second index. The rows themselves are Store.QueueState.Rows : Map<string, Job>, the payload carried as the Job.Payload string.

The in-memory Store persists these rows today; the Valkey transport EchoMQ.Connector lands next — and the same byte-order property carries to the wire, where a sorted set keyed on the id holds the order with no extra column.

The id

A fixed-width Base62 string whose bytes sort exactly as the integer does.

The order, for free

A Set or Map keyed on it is time-ordered with no sort column.

Where this leads

The key form is settled and the row's shape is fixed. The last dive puts the store behind a port: the same enqueue / claim / complete surface over an in-memory backing today, with a Valkey backing named and on the way.

↑ C4.08 · Persistence ← C4.08.1 · Keys & rows C4.08.3 · The store port →

References

Sources

FSharp.Core API reference — Set and Map, the ordered collections keyed on the id.
Microsoft — F# documentation — records, anonymous records, and structural comparison.
Microsoft — .NET documentation — String ordinal comparison, the basis of the byte order.
The F# Software Foundation — F# on .NET, the home of the echo_fs port.

Related in this course

Elixir F4.08.2 — SQLite & PostgreSQL — the parity dive: a bigint key and a time-range query.
C4.08 — Branded ids & persistence — the module hub.
C4.08.3 — The store port — the swappable backing.
C1 — F# Language — records, Set, and Map in depth.