jonnifyechomq
EchoMQ / E2 · Core / The lifecycle / The eight states

E2.01.1 · The lifecycle · dive 1

The eight states

A job is in exactly one of eight states — each one Redis key, built by one function in EchoMQ.Keys.

Three of the eight are LISTs (wait, paused, active); the other five are ZSETs, scored by run-at (delayed), priority (prioritized), finished-at (completed), or unscored (failed, waiting-children). Two are terminal: completed and failed. Pick a state on the right to read its Redis location, type, and key function.

The eight states · pick one

waitLIST pausedLIST delayedZSET · run-at prioritizedZSET · priority activeLIST · lock held w-childrenZSET · flow parent completedZSET · terminal failedZSET · terminal + stalled (SET)lock-miss set solid border = non-terminal · dashed = terminal 8 states; each one EchoMQ.Keys function

The eight, one at a time

Each state is a single Redis key. The key string is built by one function in EchoMQ.Keys, verified against keys.ex, and every key of one queue carries the hash tag emq:{queueName} so the whole queue lands on one Cluster slot.

# the eight states → their keys (EchoMQ.Keys), against emq:{q}
  wait             LIST   emq:{q}:wait              Keys.wait/1
  paused           LIST   emq:{q}:paused            Keys.paused/1
  delayed          ZSET   emq:{q}:delayed           Keys.delayed/1 · score = run-at
  prioritized      ZSET   emq:{q}:prioritized       Keys.prioritized/1 · score = priority
  active           LIST   emq:{q}:active            Keys.active/1 · lock held
  completed        ZSET   emq:{q}:completed         Keys.completed/1 · terminal · score = finished-at
  failed           ZSET   emq:{q}:failed            Keys.failed/1 · terminal
  waiting-children ZSET   emq:{q}:waiting-children  Keys.waiting_children/1

The two terminal states are completed and failed — a job there has finished its run, the latter only once its attempts are exhausted (a failed job with attempts remaining is retried into delayed instead, so it is not yet terminal). Every other state can still transition. A ninth key, the stalled SET (EchoMQ.Keys.stalled/1), is not a job state — it is the lock-miss detection set the stalled checker reads; a job is never "in" it as a location.

Where a job goes on add

A job's target state on add is decided by its options, and the add script differs per target. Pick the options below to see which state a freshly added job lands in, and which add script puts it there.

add(opts) · pick the options

add(opts) waitaddStandardJob-9 delayedaddDelayedJob-6 prioritizedaddPrioritizedJob-9 waiting-childrenaddParentJob-6

The options decide the location, and the location decides the add script. A plain job lands in wait; delay, priority, or a parent each route it elsewhere.

The Golden Rule — the state is L1

A state is a Redis key, and the key is L1 of the four-layer model — immutable and shared across the three runtimes. The same emq:{q}:wait LIST and emq:{q}:completed ZSET exist, byte-for-byte, whether the queue is driven from Elixir, Go, or Node.js; only the executor that builds the key string and reads or writes it varies.

the protocol (immutable L1)

The eight state keys — emq:{q}:wait (LIST), emq:{q}:delayed (ZSET · run-at), … — the same location and the same Redis type in every runtime.

its three runtimes (variable L3)

Elixir builds each key with EchoMQ.Keys.wait/1, active/1, … over a Redix pool; Go and Node.js build the same key strings their own way. The location and the type are the contract; the code that constructs the string is each runtime's own.

A job is in exactly one of eight states; each is one key built by one EchoMQ.Keys function, with a fixed Redis type. The state is L1 — the same location in Elixir, Go, and Node.js.

Next: every transition and its script — the atomic Lua script that moves a job between these states, and the closed -1…-11 return codes.

References

The Redis types behind the states, and the wire protocol that fixes the eight.

Sources

Related in this course