E2 · Movement I · The Core
The lifecycle, the
components & the runtimes
The as-built library in motion: the eight states a job moves through, the one atomic script behind each move, and the three runtimes that share them.
E1 froze the wire — the keys and the atomic Lua scripts. This chapter sets it in motion. A job is
always in exactly one of eight states, and every move between them is one atomic Lua script —
moveToActive-11, moveToFinished-14,
moveToDelayed-8, moveStalledJobsToWait-8 — with a closed
-1…-11 return vocabulary. Four modules build the picture: the
lifecycle and its scripts, the components that drive it (the job model, the stateless queue, the worker fetch
loop), the one-timer lock management and the supporting processes, and the three runtimes with their honest interop
status. A workshop then runs a job through every state.
Movement I · Core (as-built) ← redis-patterns R3
Why & when
A reader who walked the → EchoMQ door from the reliable-queue pattern (R3) has met
states-as-locations and the atomic-move thesis. A reader who finished E1 holds the immutable wire — the keys and
the scripts. This chapter answers the next question: which eight states a job occupies, which
script performs each transition between them, and how the lock keeps a job owned while a worker runs it.
Read it when you need the mechanism in motion — reading a real queue's state, debugging a stalled job, or
building on the core (every extension in E3–E8 stands on the lifecycle this chapter lays out).
The chapter holds one rule throughout — the Golden Rule: the state's keys (L1) and the
transition's Lua script (L2) are immutable and shared across runtimes; the executor (L3) and the API (L4) vary.
The interactive below reads the eight real states EchoMQ.Keys builds. Pick a state to
see its Redis location and the script that moves a job into it.
The eight-state lifecycle · pick a state
A job is always in exactly one state, and each is one key built by one function in
EchoMQ.Keys. The state and the script that moves a job into it are the protocol — the same
in Elixir, Go, and Node.js.
The modules
Four teaching modules take the running library apart — the lifecycle, the components, the lock management, and the three runtimes — and a workshop runs a job through every state across two runtimes.
The lifecycle & state machine
The eight states, the script behind each transition, the lock protocol, and the closed -1..-11 codes. Dives: the eight states · every transition & its script · the lock protocol.
Jobs, queues & workers
The job model and its options, the stateless queue, and the worker fetch loop with concurrency. Dives: the job model · the stateless queue · the worker fetch loop.
Lock management & the supporting processes
One timer per worker, not one per job; stalled recovery; schedulers and the events stream. Dives: one timer not N · stalled recovery · schedulers & events.
The three runtimes & interop
The Elixir library-only architecture, the honest Go atomicity gaps, and cross-runtime interop. Dives: the Elixir architecture · the Go gaps · cross-runtime interop.
Workshop
Run a job through every state — add → active → completed, and add → active → failed → retry → delayed — and observe it across two runtimes.
How it works
One transition makes the layering concrete. A worker picks up the next job: at L4 it calls a
runtime API; at L3 the executor dispatches a script by SHA
(EchoMQ.Scripts.move_to_active/4); at L2 the script
moveToActive-11 runs atomically; at L1 it moves a job id from
emq:{queue}:wait to emq:{queue}:active and sets the lock at
emq:{queue}:{jobId}:lock with a UUID token and a PX TTL. The
completion script moveToFinished-14 first verifies that token; a missing or mismatched
lock returns -2 or -6 from the closed code set. L1 and L2 are
byte-identical across runtimes; only L3 and L4 differ.
The state keys (EchoMQ.Keys.wait/1, active/1, …) and the
transition scripts (moveToActive-11, moveToFinished-14) —
frozen, with the same closed -1…-11 codes in every runtime.
Elixir dispatches the moves with EchoMQ.Scripts over a Redix pool and renews locks
with one EchoMQ.LockManager timer per worker; Go and Node.js drive the same scripts
their own way. The wire underneath does not move.
The chapter teaches the lifecycle and the components in depth, and shows, on every transition, that the runtime above is free precisely because the protocol below is fixed.
The chapter closes with E2.05 · the workshop — step a captured job through every state and read the transition script named at each edge, with the honest Go gaps noted.
Redis Patterns Applied
This chapter is the depth on the far side of the redis-patterns door. The reliable-queue lifecycle it builds in full — wait → active → completed, with stalled recovery when a worker dies — is taught as a transferable pattern, one excerpt at a time, in R3 · Reliable queues. There a reader meets the wait / active / done / recover shape once and portably; here are the eight states, the atomic script behind each move, and the lock that makes “exactly one worker” hold.
Up next — the rest of the course
E2 closes Movement I. Movement II (E3–E8) then tracks the EMQ extension ladder, each chapter on a drafted rung, teaching the redis-patterns pattern, the rung's spec, and the as-built code together.
The substrate
The opt-in floor every EMQ rung stands on. ⇄ emq.1
Groups
Fair lanes and the control plane. ⇄ emq.2
Batches
Bulk consumption with group affinity. ⇄ emq.3
Lifecycle controls
TTL, distributed cancel, and checkpoints. ⇄ emq.4
EchoCache
The branded local TTL near-cache. ⇄ emq.5
Production
Conformance, telemetry, and the benchmark gate. ⇄ emq.6
The depth chapters are specified and on the way. As each is authored, its card here links through to a built chapter; a Movement-II chapter teaches from its rung's spec until the rung ships.
References
Sources
- BullMQ — Documentation — the wire protocol EchoMQ implements: the eight states, the transition scripts, and the lock protocol.
- Redis — RPOPLPUSH — the reliable wait→active move at the heart of
moveToActive. - Redis — EVALSHA — the load-once, run-by-SHA dispatch every transition script runs under.
- Redis — Sorted sets — the ZSET scoring behind delayed, prioritized, and the terminal sets.
Related in this course
- E2.01 · The lifecycle & state machine — start here.
- E1 · The protocol & the data layer — the immutable wire this chapter sets in motion.
- redis-patterns R3 · Reliable queues — the pattern this chapter deepens.