jonnifyechomq
EchoMQ / The Protocol / The record hash / The hash and its fields

The record hash · dive 01 · the field set

The hash and its fields

One write builds the row. Its three field names are the record's shape.

The job record is one Valkey HASH, addressed by EchoMQ.Keyspace.job_key/2 at emq:{q}:job:<id> and built by exactly one HSET inside the enqueue script. The field list — state, attempts, payload — is the record's shape, fixed and named, so any speaker that reads the row knows what each field means without a schema lookup. Pick a field on the right to read its name, its birth value, and its role.

The field set The write Reading the row back References

The record · pick a field

HASH key emq:{orders}:job:JOB0f3k… state "pending" — lifecycle position attempts "0" — retry count + token payload "<caller bytes>" — opaque cargo

One hash, a fixed field set

A job is an entity, and its state is one Valkey hash. The hash holds three fields and no more at birth: there is no id field, because the id is the key; no created_at, because the branded id is itself time-ordered; no duplicate of anything. The record is the smallest row that carries a job.

What makes the field set part of the protocol, not an implementation detail, is that the names are read across the wire. When a worker claims the job, the script reads payload by that name; when a completion is fenced, the script reads attempts by that name. Any runtime that speaks the wire agrees on those three strings. The shape is the contract.

Three named fields, one hash, born from one write. The names are fixed so any speaker reads the row with no schema lookup.

The write — two beats

Beat one — the named handle. EchoMQ.Jobs.enqueue/4 builds the two keys it will touch — the job row and the pending set — and runs the @enqueue script. The row key comes from job_key/2, a gated builder that raises on an ill-formed id, so a bad id never reaches the wire.

# echo_mq — EchoMQ.Jobs
# job_key/2 addresses the row at emq:{q}:job:<id>; queue_key/2 the pending
# set. Both are built host-side and handed in declared. The values (id and
# payload) ride ARGV — never the keys.
def enqueue(conn, queue, job_id, payload) when is_binary(job_id) and is_binary(payload) do
  keys = [Keyspace.job_key(queue, job_id), Keyspace.queue_key(queue, "pending")]

  case Connector.eval(conn, @enqueue, keys, [job_id, payload]) do
    {:ok, 1} -> {:ok, :enqueued}
    {:ok, 0} -> {:ok, :duplicate}
    {:error, {:server, "EMQKIND" <> _}} -> {:error, :kind}
    other -> other
  end
end

Beat two — the script body. After the branded-id gate and the idempotency check, the record is one HSET. The three field names and their initial values are right there in the call: state = "pending", attempts = "0", payload = ARGV[2].

-- the @enqueue script — the record write, every key declared in KEYS[]
-- The branded-id gate: the id must be JOB-namespaced, else refused at the wire.
if string.sub(ARGV[1], 1, 3) ~= 'JOB' then
  return redis.error_reply('EMQKIND job id must be JOB-namespaced')
end
-- Idempotent: a row that already exists is a no-op (a duplicate enqueue).
if redis.call('EXISTS', KEYS[1]) == 1 then
  return 0
end
-- THE FIELD SET: one HSET, three named fields, in field/value pairs.
--   'state'    'pending'  — born at the head of the lifecycle
--   'attempts' '0'        — born with zero tries
--   'payload'  ARGV[2]    — the caller's bytes, stored verbatim
redis.call('HSET', KEYS[1], 'state', 'pending', 'attempts', '0', 'payload', ARGV[2])
-- The pending set: the id is the member, so byte order is mint order.
redis.call('ZADD', KEYS[2], 0, ARGV[1])
return 1

The HSET is the field set, literally. Lifecycle transitions add a small, fixed set of further fields only when they apply — a failed retry writes last_error, the progress verb writes progress, a lane member carries group — but the row a job is born with is exactly these three.

Reading the row back

The same three names are what a reader looks for. A reader naming a field that exists gets its value; a reader naming a field that is not part of the record — or misspelling one — gets nothing, because the protocol is the exact strings. Read a field off the sample row and see what comes back.

HGET the sample row · which field name?

the pattern (Redis Patterns Applied)

Model an entity as one Valkey hash with a small, fixed field set — the key is the address, the fields are the contract. Redis Patterns Applied · Data Modeling & Memory (R7) teaches the move; Redis Patterns Applied opens the door and Patterns become protocol is the near side.

the implementation (echo_mq)

job_key/2 addresses the row; @enqueue's HSET fixes the three field names. A reader that names payload gets the cargo; a reader that names data finds nothing.

The record is the exact field strings. Name payload and the row answers; name data and it is silent — the names are the protocol.

Recap

A job is a Valkey hash at emq:{q}:job:<id> with three named fields written by one atomic HSET in @enqueue: state, attempts, payload. The names are read across the wire, so the field set is the contract, not an implementation detail. The next dive reads two of those fields in depth: the lifecycle field and the fencing token.

Next in this module: The state and attempts — the lifecycle position the transitions rewrite, and the retry count that doubles as the worker's fencing token.

References

Where the record's data type and the writes that build and read it are documented.

Sources

Related in this course