jonnifyechomq

Immutability · dive 01 · why

The immutable line

An agreement only holds when both sides hold it byte-for-byte. Below the line, both sides do.

Below the language line are two layers — L1, the keys and field names, and L2, the atomic Lua. They are fixed, and not by policy: every speaker reads the same key at the same address and the same named fields out of the same row. Rename one of them in one speaker and that speaker writes a row no other speaker can find. This dive deepens the line on the real record — the key EchoMQ.Keyspace.queue_key/2 builds, and the three fields the @enqueue script writes — and shows exactly who breaks for each thing you change.

Change one thing · who can still read the job?

below the line
above the line
three speakers, one row Elixircanonical a Go peera runtime a Node peera runtime emq:{q}:job:<id> state · attempts · payload

An agreement is only as fixed as both sides keep it

The three pillars interoperate and any runtime can be a peer for one reason: the agreement is the keys and the scripts. But an agreement only holds if both sides hold it byte-for-byte. That is what "the line is fixed" means — not that the code cannot be changed, but that changing the keys or the field names in one speaker desyncs that speaker from every other. The data is not destroyed; it becomes unfindable (a moved key) or unreadable (a renamed field) across the wire.

The demonstrator above makes the consequence exact. Three speakers — the canonical Elixir and two peer runtimes — all read one row at emq:{q}:job:<id> with the fields state, attempts, payload, through a script whose KEYS are the row and the pending set. Make one change and read how many of the three can still find and read the job. A change below the line breaks the changer; a change above it — the L4 verb — touches nothing on the wire.

The line, on the real record

The address is L1, and it is built by one function. EchoMQ.Keyspace.queue_key/2 composes the exact bytes every speaker must agree on: the literal emq:, the queue name in braces, the type suffix.

# echo_mq — EchoMQ.Keyspace
# The literal "emq:", the queue name wrapped in braces, then the type
# suffix. Every speaker composes the exact same bytes for the same
# {queue, type} — that byte-for-byte agreement is the line.
def queue_key(queue, type) when is_binary(queue) and is_binary(type),
  do: IO.iodata_to_binary(["emq:{", queue, "}:", type])

The field names are L1 too, and they are written by the @enqueue script (L2) in one atomic HSET. Three names, fixed: state is the lifecycle position, attempts the retry fence, payload the opaque caller bytes. A reader anywhere on the wire reads exactly these three.

-- the @enqueue script (L2) — the three-field row, written atomically
-- These three names ARE the contract. Every speaker looks up exactly
-- state, attempts, payload — no more, no fewer, by these names.
redis.call('HSET', KEYS[1], 'state', 'pending', 'attempts', '0', 'payload', ARGV[2])

Now the consequence is concrete. Change the literal emq: in one runtime and its rows land at an address no other speaker scans — the jobs are not lost, they are unfindable. Rename payload to data in that runtime's @enqueue and it writes a field nobody else reads — a claim returns the row, but every other speaker reads payload and finds nothing. The same logic applies to the KEYS contract: drop the pending set from the declared keys and the row is written but never enters the queue, so nothing ever claims it.

The L4 verb is the mirror image. EchoMQ.Jobs.enqueue/4 is a name above the line; rename it to push and nothing on the wire moves, because the keys it builds and the script it runs are unchanged. That asymmetry — fixed below, free above — is the whole reason one wire serves three pillars in any language.

Below the line is the contract; above it is the code. A field rename below the line makes the row invisible to every other speaker; an L4 rename above it changes nothing on the wire.

The layer ledger — below or above, fixed or free

Read EchoMQ as five layers and mark each one. The two below the line are fixed for every runtime; the two above it are each runtime's own; the engine underneath is external. Pick a layer to read where it sits and what "fixed" or "free" means for it in practice.

The layers · pick one to read its standing

L4 API · EchoMQ.Jobs.enqueue L3 executor · EVALSHA, the SHA cache — fixed below this line — L2 Lua · the atomic state machine L1 keys & fields · emq:{q}:<type>, state/attempts/payload L0 Valkey · the engine (external)

The full layer model — and the stack drawn end to end — is the orientation dive The protocol below the line. This dive reads the two fixed layers on the real record.

Pattern & implementation

the pattern (Redis Patterns Applied)

A Redis convention becomes a protocol when the agreement — the keys plus the atomic scripts — is pushed below the language line and held fixed. Coordination teaches the atomic, one-slot move that this fixity rests on.

the implementation (echo_mq)

EchoMQ.Keyspace.queue_key/2 fixes the address emq:{q}:<type>; the @enqueue script fixes the row with the three named fields and every key declared in KEYS[]. L1 and L2, shared and fixed.

Push the agreement below the language line and hold it byte-for-byte, and a Redis convention becomes a protocol any runtime can be a peer of.

Recap — the line, and who keeps it

The line is the keys and the Lua, and it is fixed because an agreement only holds when both sides hold it byte-for-byte. EchoMQ.Keyspace.queue_key/2 builds the address emq:{q}:<type>; the @enqueue script writes the three named fields state, attempts, payload. A change below the line makes the row unfindable or unreadable to every other speaker; the L4 verb above the line is free to change. The next dive reads the second guard — how identity is gated before a row is even written.

Next in this module: The branded-id gate — the 14-byte branded id and the EMQKIND Lua check that admits only JOB-namespaced ids.

References

Where the line is grounded, and where the keyspace and record layers are documented.

Sources

Related in this course