The state and attempts

One field says where the job is. One field counts the tries — and fences the work.

Two of the three record fields carry the job's machinery. state is the lifecycle position — one field holds where the job is, and each atomic transition rewrites it. attempts is the retry count and, at the same time, the fencing token: claim increments it and hands the new value to the worker, and every later transition refuses a call whose token no longer matches. Pick a transition on the right to read the state it writes.

state — the lifecycle attempts — the fence The fencing check References

The state field · pick a transition

state — the lifecycle, in one field

The state field holds where the job is, and nothing else holds it. There is no separate status table, no set membership that has to agree with a flag: the one field is the answer, and each atomic transition rewrites it as part of the same script that moves the job between sets.

@enqueue writes "pending"; @claim writes "active"; @schedule writes "scheduled" for a delayed or backed-off job; @retry writes "dead" at the attempt ceiling or "scheduled" for another try; reprocess returns a "dead" job to "pending". Because the field is written inside the transition's own script, the position can never disagree with where the job actually is.

One field is the lifecycle position. Each transition rewrites it atomically, so the field and the job's real place never drift.

attempts — the retry count and the token

The attempts field counts how many times the job has been claimed, and the same counter is the worker's fencing token. Beat one is the named handle: EchoMQ.Jobs.claim/3 pops the oldest pending id, then runs the @claim script.

# echo_mq — EchoMQ.Jobs
# claim pops the oldest pending id and runs @claim. KEYS = [pending, active];
# ARGV carries the queue base and the lease. The script returns {id, payload,
# attempts} — the attempts value IS the worker's fencing token for this lease.
def claim(conn, queue, lease_ms) when is_integer(lease_ms) and lease_ms > 0 do
  keys = [Keyspace.queue_key(queue, "pending"), Keyspace.queue_key(queue, "active")]
  argv = [Keyspace.queue_key(queue, "job:"), Integer.to_string(lease_ms)]

  case Connector.eval(conn, @claim, keys, argv) do
    {:ok, []} -> :empty
    {:ok, [id, payload, att]} -> {:ok, {id, payload, att}}
    other -> other
  end
end

Beat two is the script body. @claim increments attempts with HINCRBY — the returned value is the token — sets state to "active", leases the id on the active set at the server clock, and returns the id, the payload, and the new attempts.

-- the @claim script — mint the token, lease on the server clock
-- Pop the oldest pending id; an empty set ends the claim.
local popped = redis.call('ZPOPMIN', KEYS[1])
if #popped == 0 then return {} end
local id = popped[1]
local jk = ARGV[1] .. id
-- HINCRBY mints the token: the new attempts value fences this lease.
local att = redis.call('HINCRBY', jk, 'attempts', 1)
-- The lifecycle field moves to "active" in the same atomic step.
redis.call('HSET', jk, 'state', 'active')
-- The lease is the active-set score: now + lease_ms, on the SERVER clock.
local t = redis.call('TIME')
local now = t[1] * 1000 + math.floor(t[2] / 1000)
redis.call('ZADD', KEYS[2], now + tonumber(ARGV[2]), id)
return {id, redis.call('HGET', jk, 'payload'), att}

So the retry count and the lease generation are the same number. If a worker stalls past its lease, the reaper returns the job to pending; the next claim bumps attempts again, minting a new token. The first worker's token is now stale — one field, two jobs.

attempts is the retry count and the fencing token at once: claim mints it with HINCRBY, and a new claim invalidates the old token.

The fencing check

Completion is token-fenced. @complete reads attempts off the row and compares it to the token the worker holds; a mismatch is refused with the EMQSTALE wire class, so a worker whose lease expired cannot complete a job another worker now owns. Run the check against the sample row, whose current token is 2.

complete with a token · the row's attempts is 2

the pattern (Redis Patterns Applied)

Carry control state in the entity's own fields, and let one counter serve two roles when the roles are one fact — the attempt count and the lease generation are the same number. Redis Patterns Applied · Data Modeling & Memory (R7) teaches the move; Redis Patterns Applied opens the door and Patterns become protocol is the near side.

the implementation (echo_mq)

state is the lifecycle position the transitions rewrite; attempts is the retry fence and the token @claim mints with HINCRBY and @complete checks with att ~= ARGV[2].

The token is the row's attempts. A current token completes; a stale one is refused EMQSTALE — the field that counts tries is the field that fences them.

Recap

state holds the job's lifecycle position, rewritten by each atomic transition so the field never drifts from where the job is. attempts is the retry count and the fencing token: @claim mints it with HINCRBY, and @complete refuses a stale token with EMQSTALE. Two fields carry the machinery; the third is the cargo, the next dive's subject.

Next in this module: The payload — the caller's opaque cargo, stored and returned verbatim and never read by the wire.

References

Where the writes behind the lifecycle field and the fencing token are documented.

Sources

Valkey — HINCRBY — the atomic increment that mints the fencing token.
Valkey — HSET — the write each transition uses to set state.
Valkey — Documentation — the substrate of record.

Related in this course

The record hash — the module hub.
The hash and its fields — the previous dive.
The payload — the next dive.
The Protocol — the chapter landing.