States as locations

A naive queue stores a status column on the job record — "wait", "active", "done" — and updates it as the job moves. Two facts can then disagree: where the job actually sits and what its status claims. A crash between writing the status and moving the job leaves the pair out of sync, and nothing reconciles them.

EchoMQ holds no status field. A job's state is the Redis key its id lives in. The wait and active states are LISTs; delayed, prioritized, completed and failed are sorted sets scored by time or priority:

# a job's state is the key its id lives in — no status column
emq:{emails}:wait         → LIST  · ids waiting to run
emq:{emails}:active       → LIST  · ids in flight under a lock
emq:{emails}:delayed      → ZSET  · scored by when the job is due
emq:{emails}:prioritized  → ZSET  · scored by priority
emq:{emails}:completed    → ZSET  · finished ids, scored by finish time
emq:{emails}:failed       → ZSET  · failed ids

EchoMQ.Keys builds each of these from the queue context — wait/1, active/1, delayed/1, prioritized/1, completed/1, failed/1. A transition is moving the id from one key to the next. There is no separate state column to fall out of sync, because the location is the state. The hero figure runs exactly this — fire a transition and the id hops to its new key.

Moving a job from active to completed is not one write. The transition reads the job, checks its lock, removes the id from active, writes the result into completed, updates counters and metrics, and pushes the next job — touching many keys. Done as separate commands, a crash partway leaves the job in two places, or in none.

moveToFinished-14.lua performs that whole transition across fourteen keys in one EVALSHA. Its header documents the fourteen KEYS, and EchoMQ.Scripts.move_to_finished/7 assembles them in order:

# moveToFinished-14.lua — the 14 keys of one lifecycle transition
KEYS[1]  wait          KEYS[8]   paused
KEYS[2]  active        KEYS[9]   meta
KEYS[3]  prioritized   KEYS[10]  pc (priority counter)
KEYS[4]  events        KEYS[11]  completed / failed
KEYS[5]  stalled       KEYS[12]  job hash
KEYS[6]  limiter       KEYS[13]  metrics
KEYS[7]  delayed       KEYS[14]  marker

Redis runs a Lua script to completion with no other client interleaving, so the move applies in full or not at all. There is no torn intermediate state to recover. This is the R2.01 atomic-update pattern at lifecycle scale: read, decide, write — in one move. The figure below contrasts that one script against the same lifecycle done as separate commands, and shows where a crash leaves the job torn.

The script is the protocol, and it is cached by its SHA1. EchoMQ.Scripts.execute_raw/4 computes the script's sha1, then issues EVALSHA sha numkeys keys… args…. On the first call for a script Redis has not yet seen, it answers NOSCRIPT; execute_raw/4 matches that error, falls back to EVAL with the full body (which also caches it), and every later call hits EVALSHA:

# EchoMQ.Scripts.execute_raw/4 — run by SHA, fall back to EVAL on NOSCRIPT
sha = sha1(script)
case command(conn, ["EVALSHA", sha, num_keys | keys ++ args]) do
  {:ok, result} -> {:ok, result}
  {:error, %Redix.Error{message: "NOSCRIPT" <> _}} ->
    command(conn, ["EVAL", script, num_keys | keys ++ args])   # caches it too
end

The full roughly thousand-line moveToFinished-14.lua body crosses the wire once. After that the worker sends only its forty-character SHA and the keys and arguments. The script lives in Redis; the caller references it by hash.

A worker that loops LRANGE over the wait list and sleeps burns CPU and adds latency between a job arriving and a worker noticing it. EchoMQ wakes a blocked worker instead. Every enqueue touches the emq:{queue}:marker ZSET (EchoMQ.Keys.marker/1), and an idle worker parks on BZPOPMIN marker timeout on a dedicated blocking connection:

# worker.ex do_wait_for_job/3 — sleep until a marker says a job arrived
marker_key = Keys.marker(state.keys)
Redix.command(blocking_conn, ["BZPOPMIN", marker_key, timeout_seconds])
# returns [key, member, score] when a job arrives, or nil on timeout

The worker sleeps until a job arrives; then Redis returns the popped marker and the worker fetches the job. No CPU is spent while the queue is empty, and pickup latency is near zero rather than half a poll interval. The Go port's polling ticker is the contrast — it wakes on a fixed interval whether or not a job is waiting.

The pattern and the application line up directly. A job's state is its Redis location, every transition is one atomic Lua move, and an idle worker blocks on a marker rather than polling. In EchoMQ that is moveToFinished-14.lua — fourteen keys, one EVALSHA via EchoMQ.Scripts.execute_raw/4 — together with emq:{queue}:marker and BZPOPMIN. Each piece is real code in echo/apps/echomq: EchoMQ.Keys for the state keys, EchoMQ.Scripts for the move, EchoMQ.Worker for the blocking pickup.

Where the atomic lifecycle script is run by hash, the cached-script protocol it relies on, the blocking pop that wakes a worker, and the reliable-queue protocol whose Lua scripts are the protocol.