jonnify·redis /redis-patterns/overview/patterns-become-protocol/the-immutable-core

R0.3.2 · Patterns become protocol — dive 2

The immutable core.

The core of the protocol is small enough to hold in one hand: a closed key grammar, a reserve of exactly four deployment keys, a job row of exactly three fields, and four sorted sets. What keeps it stable is not a pinned upstream commit — it is governance the protocol carries itself.

Grounding: content/bcs3.1.md (the grammar and the fence) · bcs3.2.md (the row and the enqueue script) · bcs3.3.md (the four sets and the fencing token) · bcsA.md · docs/echo_mq/emq.design.md.

Every key parses. Select or hover a key; the readout names its class in the grammar and what the structure carries.
emq:{orders}:pending emq:{orders}:active emq:{orders}:schedule emq:{orders}:dead emq:{orders}:job:ORD0NgWEfAEJfs 17 bytes of grammar 14 bytes of identity {emq}:version
readout · Select or hover a key. Any key in the system answers to one of three classes: a per-queue type, the gated job position, or a member of the {emq}: reserve.

Static verdicts — pending → score-zero ZSET, lex order is mint order · active → lease-scored ZSET, the expiry index · schedule → run-at-scored ZSET · dead → score-zero ZSET, the morgue · the job row → a three-field HASH, 17 bytes of grammar before a 14-byte branded payload · {emq}:version → the reserve’s canonical tenant. Per-queue keys begin emq:{; reserve keys begin {emq}: — disjoint at the first byte.

§1

The closed grammar and the reserve

content/bcs3.1.md · emq.md

Every key in the system parses. Per-queue keys are emq:{q}:<type>. Job rows compose the grammar with the identity canon: emq:{orders}:job:ORD0NgWEfAEJfs is seventeen bytes of grammar before a fourteen-byte branded payload. And the braced base is reserved for facts about the deployment itself: at the current wire version the {emq}: reserve holds exactly four members — {emq}:version, {emq}:locks, {emq}:bundle, {emq}:migration:<q>. A per-queue key begins emq:{ and a reserve key begins {emq}: — disjoint at the first byte, so any key classifies by parse.

Two laws ride the grammar. The hashtag is the queue name: every key of one queue answers one cluster slot, so every per-queue transition script is single-slot legal by grammar. And wellformedness lives at the key, policy at the script: the key builder refuses anything that does not parse as a fourteen-byte branded id, while kind policy — which namespaces a queue admits — belongs to the enqueue script.

content/bcs3.1.md · the grammar, shape by shape · committed linefrozen
emq:{orders}:pending | emq:{orders}:job:ORD0NgWEfAEJfs | {emq}:version | {emq}:locks -- 17 bytes before the payload
§2

The row, the sets, and the admission law

content/bcs3.2.md · bcs3.3.md

A job is a hash of three fields — state, attempts, payload — and deliberately nothing more. No enqueued_at exists because mint time already lives inside the identity; server-time facts belong to leases. The four sorted sets each earn their score semantics: pending stays score-zero forever, so equal-score lex order is mint order — the set is the FIFO, the browse index, and the time-range index at once; active is scored by lease deadline, so it is simultaneously the in-flight roster and the expiry index; schedule is scored by run-at — the separate set that keeps mixed scores out of the lex law; dead is score-zero again, so the morgue browses newest-first like everything else.

Admission is one idempotent script, and the order of its three acts is a decision — policy before existence before write:

content/bcs3.2.md · the enqueue script · verbatimfrozen
if string.sub(ARGV[1], 1, 3) ~= 'JOB' then
  return redis.error_reply('EMQKIND job id must be JOB-namespaced')
end
if redis.call('EXISTS', KEYS[1]) == 1 then
  return 0
end
redis.call('HSET', KEYS[1], 'state', 'pending', 'attempts', '0', 'payload', ARGV[2])
redis.call('ZADD', KEYS[2], 0, ARGV[1])
return 1

The committed line for the refusal reads: an ORD id in the job position answers EMQKIND on the wire -- the key let it pass, the law did not. Duplicates are a success shape — first call enqueued, second answered duplicate, the row untouched and pending holds 1 — because an at-least-once producer retries on any doubt and the id is the receipt.

Notes on ValkeyThe admission lands whole because script execution is atomic — “all of the script’s effects either have yet to happen or had already happened” — valkey.io/topics/programmability.

§3

Governance — frozen by fence, not by pin

bcsA · emq.md

What keeps the core immutable is the protocol’s own machinery, in three parts.

The fence. At every connect the connector reads {emq}:version, claims it on a fresh keyspace, verifies the read-back, and refuses any other value with a typed error — {:error, {:version_fence, got}} at boot, a stop the supervisor records on reconnect. The committed line reads: GET {emq}:version answers echomq:2.0.0 through the fenced connector itself — the read travels through a connection that could not exist had the fence not held.

The change rules. The wire version is recorded monotonically and changes only under written rules: additive registration — a new key type, a new reserve member with its probe, a new wire class with its probe — is a minor; a wire break or a field repurpose is a major. The break happened exactly once: EchoMQ broke from its v1 line, now frozen at 1.3.0.

The wire classes. Every refusal a bundle script issues leads with its class word — a closed registry of two: EMQKIND (kind refusal at enqueue) and EMQSTALE (fencing-token refusal). A refusal that names its class is parseable policy, not a stringly accident.

the application · the fence, on the wire

A typed boot check, gated live

The fence is exercised in the committed connector gate — the claim, the planted mismatch, and the supervised restart:

content/bcsA.md · the fence, on the wire · against live Valkeyfrozen
fence claimed                 echomq:2.0.0   ← on a fresh keyspace
planted mismatch              refused        ← typed {:version_fence, got}, fatal at boot
supervised restart            re-fenced, served
gate record                   PASS 8/8
§4

The fencing token

content/bcs3.3.md · verbatim

The protocol’s quiet center is one integer doing two jobs. The claim script increments attempts with HINCRBY — monotonic per job by construction, because only claim increments it — and every later transition verifies the token before touching anything:

content/bcs3.3.md · the verification half · verbatimfrozen
local att = redis.call('HGET', KEYS[2], 'attempts')
if att ~= ARGV[2] then return redis.error_reply('EMQSTALE complete token mismatch') end
The token, performed. The row’s attempts counter reads 1 — minted by the claim. Pick a transition and a presented token; the readout answers as the wire answers.
readout · complete with token 1: the right token retires the row — nothing remains.

Static verdicts — the right token retires or parks the row; a stale token earns EMQSTALE and the lease holder’s work survives the zombie’s complete; across retry → promote → claim, the next token is 2 — one job, two lives, one counter.

the pattern

Write the convention down exactly, version it, and let the protocol police itself — typed refusals on the wire instead of trust between processes.

its EchoMQ application

A closed grammar, a four-member reserve, a three-field row, and four sets — held still by the two-way fence, the written change rules, and EMQKIND / EMQSTALE.

The core is immutable by construction: a fence that refuses at boot, change rules that are written down, and one counter that both counts lives and fences them.

§5

References

sources · related

Sources

Related in this course