jonnifyechomq
EchoMQ / The Protocol / The record hash / The payload

The record hash · dive 03 · the cargo

The payload

The third field is the caller's. The wire stores it, moves it, delivers it — and never looks inside.

The third field, payload, is the caller's data, and the wire never reads inside it. Enqueue takes it as ARGV[2] and stores it verbatim; claim hands it back unchanged. Between those two points the protocol does nothing with it — it is opaque cargo, addressed, moved, and delivered but never parsed, validated, or branched on. That is what lets one wire carry any job for any consumer. Round-trip a sample payload on the right and read it back, byte for byte.

Enqueue then claim · pick a payload

@enqueue · HSET payload ← ARGV[2] @claim · HGET payload → worker verbatim in: … out: …

Opaque cargo

The payload is the caller's, and the protocol holds no opinion about its bytes. It is not parsed on the way in, not validated, not indexed, not branched on; it is stored as one string and returned as the same string. The system's job is to address it, move it through the lifecycle, and deliver it to a worker — not to understand it.

That is deliberate, and it is what makes the wire general. Because the protocol carries the value without reading it, the same queue serves any job for any consumer, in whatever encoding the caller and the worker agree on — JSON, a bare string, a packed binary. The cargo law: the bus carries; the caller decides what.

The payload is opaque cargo: stored, moved, delivered, never understood. The wire holds no opinion about the bytes, so it carries any job.

Store and return — two beats

Beat one — the named handle. EchoMQ.Jobs.enqueue/4 requires the payload to be a binary and passes it as ARGV[2]. The guard makes the field always hold a string; the contents stay the caller's.

# echo_mq — EchoMQ.Jobs
# The is_binary(payload) guard means the field always holds a string — but its
# CONTENTS are the caller's. The payload rides ARGV[2]; the wire never reads it.
def enqueue(conn, queue, job_id, payload) when is_binary(job_id) and is_binary(payload) do
  keys = [Keyspace.job_key(queue, job_id), Keyspace.queue_key(queue, "pending")]

  case Connector.eval(conn, @enqueue, keys, [job_id, payload]) do
    {:ok, 1} -> {:ok, :enqueued}
    {:ok, 0} -> {:ok, :duplicate}
    {:error, {:server, "EMQKIND" <> _}} -> {:error, :kind}
    other -> other
  end
end

Beat two — the two scripts. @enqueue stores the payload with HSET, untouched; @claim returns it with HGET, untouched. Side by side, the round trip is store-then-return — the wire reads neither end of the value.

-- @enqueue (the write): the payload is ARGV[2], stored verbatim as the field.
redis.call('HSET', KEYS[1], 'state', 'pending', 'attempts', '0', 'payload', ARGV[2])

-- @claim (the read): the worker is handed HGET payload, byte-for-byte the same.
-- The return is {id, payload, attempts} — the payload is read, not interpreted.
return {id, redis.call('HGET', jk, 'payload'), att}

Nothing between the HSET and the HGET looks at the value. The protocol's only contract over the record is the three field names and the meanings of state and attempts; payload is a passthrough.

Opaque vs protocol — which fields the wire reads

Two of the three fields the wire reads and acts on; one it only carries. state and attempts are protocol fields — a script reads them and branches: @complete compares attempts to the token, @claim sets state. payload is the opaque field — a script moves it and never reads it. Pick a field and see which the system interprets.

Does the wire read this field?

the pattern (Redis Patterns Applied)

Keep the payload opaque to the store: let the value be the caller's, and the key and the control fields be the system's. Redis Patterns Applied · Data Modeling & Memory (R7) teaches the move; Redis Patterns Applied opens the door and Patterns become protocol is the near side.

the implementation (echo_mq)

payload is ARGV[2] stored verbatim by @enqueue and returned verbatim by @claim; the wire reads only state and attempts. The cargo law, in the script.

The wire reads state and attempts and acts on them; payload it only carries. The value is the caller's, so one wire fits any job.

Recap

payload is the caller's opaque cargo — stored verbatim by @enqueue, returned verbatim by @claim, never read by the wire. The protocol interprets the key and the control fields state and attempts; the value is the caller's. That is what lets one wire carry any job. With all three fields read, the record is complete — the chapter's remaining modules hold this shape fixed with the Lua layer and the immutability discipline.

Back to The record hash hub, or on to the rest of The Protocol — the Lua layer and the immutability that hold the field set fixed.

References

Where the writes that store and return the payload are documented.

Sources

Related in this course