jonnifyechomq

The Protocol · module 02

The record hash

A job is one Valkey hash, and its field names are part of the wire.

A job is an entity. Its state lives in one Valkey HASH at the key the keyspace builds — EchoMQ.Keyspace.job_key/2 → emq:{q}:job:<id> — and the enqueue script writes it in one atomic step. Three fields: state, attempts, payload. They are not implementation detail; they are the protocol, read by name wherever the wire runs. This module reads the record — the field set, the lifecycle field and the fencing token, and the opaque cargo.

The record The write The dives References

One hash, three named fields

The record is the smallest row that carries a job: a hash with three fields, addressed by the job key. The names are fixed — state for where the job is in its lifecycle, attempts for the retry count, payload for the caller's data. There is no id field (the id is the key) and no timestamp field; the id itself is time-ordered. Pick a field to read what it holds and which transition writes it.

The record at emq:{q}:job:<id> · pick a field

HASH key emq:{orders}:job:JOB0f3k… state "pending" — lifecycle position attempts "0" — retry count + token payload "<caller bytes>" — opaque cargo

Three fields, one hash: where the job is, how many tries it has had, and what it carries. The names are the contract.

The write — one atomic HSET

The record is created by exactly one write, inside the enqueue transition. Beat one is the named handle: EchoMQ.Jobs.enqueue/4 builds the two keys — the job row (job_key/2) and the pending set (queue_key/2) — and runs the @enqueue script through the connector.

# echo_mq — EchoMQ.Jobs
# A job is an entity: its row is a hash at the job key, the pending set is a
# same-score sorted set keyed by the id itself. The host builds both keys and
# hands them in declared; the values (id, payload) ride ARGV.
def enqueue(conn, queue, job_id, payload) when is_binary(job_id) and is_binary(payload) do
  keys = [Keyspace.job_key(queue, job_id), Keyspace.queue_key(queue, "pending")]

  case Connector.eval(conn, @enqueue, keys, [job_id, payload]) do
    {:ok, 1} -> {:ok, :enqueued}
    {:ok, 0} -> {:ok, :duplicate}
    {:error, {:server, "EMQKIND" <> _}} -> {:error, :kind}
    other -> other
  end
end

Beat two is the script body. After the branded-id gate and the idempotency check, the record is one HSET: the three fields, with their initial values. state is born "pending", attempts is born "0", and payload is the caller's ARGV[2], stored verbatim.

-- the @enqueue script — the record write, every key declared in KEYS[]
-- The branded-id gate: the id must be JOB-namespaced, else refused at the wire.
if string.sub(ARGV[1], 1, 3) ~= 'JOB' then
  return redis.error_reply('EMQKIND job id must be JOB-namespaced')
end
-- Idempotent: a row that already exists is a no-op (a duplicate enqueue).
if redis.call('EXISTS', KEYS[1]) == 1 then
  return 0
end
-- THE RECORD: one HSET writes the three named fields with their birth values.
--   state    "pending"  — the lifecycle position
--   attempts "0"        — the retry count (and, once claimed, the fencing token)
--   payload  ARGV[2]    — the caller's opaque cargo, stored verbatim
redis.call('HSET', KEYS[1], 'state', 'pending', 'attempts', '0', 'payload', ARGV[2])
-- The pending set: the id is the member, so byte order is mint order.
redis.call('ZADD', KEYS[2], 0, ARGV[1])
return 1

That HSET is the record, literally. The same three field names are what claim reads back and what every other speaker on the wire looks for. Lifecycle transitions add a small, fixed set of further fields when they apply — last_error when a retry dead-letters, progress when the progress verb runs, group for a lane member — but the row a job is born with is these three.

the pattern (Redis Patterns Applied)

Model an entity as one Valkey hash with a small, fixed field set: the key is the address, the fields are the contract. Redis Patterns Applied · Data Modeling & Memory (R7) teaches the move; Redis Patterns Applied opens the door and Patterns become protocol is the near side.

the implementation (echo_mq)

EchoMQ.Keyspace.job_key/2 addresses the row at emq:{q}:job:<id>; the @enqueue script fixes its field set — state, attempts, payload — written atomically with one HSET.

The record is one HSET: three named fields, born with their values. The names are protocol; the shape is the contract every speaker reads.

The dives

Three readings of the record, each grounded in the real echo/apps/echo_mq scripts.

built page ready to read
01built

The hash and its fields

The HSET in the @enqueue script, field by field: state, attempts, payload — the named shape of the row.
02built

The state and attempts

state, the lifecycle position the transitions rewrite; attempts, the retry count and the fencing token.
03built

The payload

The opaque cargo: stored verbatim by @enqueue, returned verbatim by @claim, never read by the wire.

Next module in The Protocol: the chapter map — the Lua layer and the immutability discipline that hold the field set fixed.

References

Where the record's data type and the writes that build and read it are documented.

Sources

Related in this course