The in-flight list

The in-flight list is not a buffer jobs pass through. It is a ledger: at any instant it holds exactly the jobs being processed right now.

A job moved into active stays there for the whole duration of the work, and only an explicit acknowledgement removes it. So the list is a live record of in-flight work — and a worker that dies mid-job leaves its job sitting in the record, found, attributed, and returnable. Per-worker lists name the holder; the LREM active 1 job ack runs only when the work is truly done. Toggle a worker dead on the right and watch its job stay in active, recoverable.

A record, not a buffer Per-worker lists The ack, when done In EchoMQ References

The recovery ledger · a dead worker leaves its job in active

The list is a record, not a buffer

A job moved into active does not pass through — it stays there for the entire duration of the work, and only an explicit acknowledgement removes it. So active is a live record: every job in it is a job a worker has taken and not yet finished. Reading the list answers a question that matters for reliability — what work is in flight right now — without any extra bookkeeping.

emq:{queue}:active  (LIST)   # -> [ job 7, job 6 ]   two jobs in flight

This is what makes a crash recoverable. When a worker dies between taking a job and acknowledging it, the job is still in active. Nothing was lost, because nothing removed it. A recovery process reads the list and finds the parked job, exactly because the list is a record of in-flight work and not a transient buffer.

The in-flight list is a live record of work in progress; a crashed worker leaves its job in the record, recoverable, because only an explicit ack removes it.

Per-worker lists attribute the parked job

One shared in-flight list works, but it cannot distinguish which worker holds which job. Giving each worker its own in-flight list does: processing:worker1, processing:worker2. A job in processing:worker1 is held by worker 1. When worker 1 dies, a recovery monitor reads worker 1's list, finds the parked jobs, and returns them to the waiting list for another worker — LMOVE processing:worker1 work_queue RIGHT RIGHT. Per-worker lists make the attribution direct: the list name is the worker.

processing:worker1  (LIST)   # jobs held by worker 1
processing:worker2  (LIST)   # jobs held by worker 2

A real engine refines the death signal beyond "how long has the job sat here." EchoMQ marks each in-flight job with a lock that has a TTL, renewed by the live worker on a heartbeat; an expired lock is the death signal, and a stalled sweep returns the job to the waiting list. R3.03 is that sweep. Here the shape is enough: a per-worker list attributes a parked job to the worker that left it.

per-worker list: Each worker has its own in-flight list (processing:workerN), so a recovery monitor attributes a parked job to the worker that holds it.
recovery ledger: The in-flight list read as a record: exactly the jobs in flight per worker, found and returnable after a crash.
expired lock: The death signal a real engine uses instead of elapsed time: a live worker renews the lock; an expired lock means no worker holds the job.

Give each worker its own in-flight list so the list name is the holder; a recovery monitor reads a dead worker's list and returns its parked jobs to wait.

The ack removes it only when done

A job leaves the in-flight list with one command: LREM active 1 job — remove one occurrence of this job from the list. The discipline is when it runs: only after the work is truly complete. Run it too early — before the side effect is durable — and a crash after the ack but before the effect loses the job for real, because the ledger no longer holds it. Run it after the effect, and a crash before the ack leaves the job parked, recoverable, and redelivered.

job = RPOPLPUSH wait active   # take the job into the in-flight ledger
process(job)                  # do the work; durable side effect lands here
LREM active 1 job             # ack: remove from the ledger ONLY now

That ordering is what makes delivery at-least-once rather than at-most-once. A job is delivered again if the worker died before the ack, and delivered once if the ack landed — never zero times. The cost is that a worker which finished the work and died before the ack redelivers the job, which is why R3.02 makes the consumer idempotent. This dive's job is the ledger and the ack; the redelivery cost is the next module's. Slide the crash point below and watch where the ack ordering protects the job.

Where the crash lands · ack-after-done protects the job

The ack is one LREM, and its timing is the rule: remove the job only after the work is durable, which is what makes delivery at-least-once.

In EchoMQ — the in-flight ledger under a lock

EchoMQ's in-flight list is emq:{queue}:active, built by EchoMQ.Keys.active/1 ("#{base(ctx)}:active"), real in echo/apps/echomq. A job is moved into it under a lock named by EchoMQ.Keys.lock/2 (emq:{queue}:<id>:lock, "#{job(ctx, job_id)}:lock"), so the parked job is both in the ledger and attributable to its worker. A crash leaves the job in active with its lock still set; once the lock's TTL expires, the stalled sweep returns it to wait. The ack that removes a finished job from active runs only after the work is done.

-- moveToActive-11.lua — the job lands in the in-flight ledger, under a lock (real)
local activeKey = KEYS[2]    -- emq:{queue}:active  (EchoMQ.Keys.active/1)
local jobId = rcall("RPOPLPUSH", waitKey, activeKey)  -- job parked in the ledger
-- the lock emq:{queue}:<id>:lock (EchoMQ.Keys.lock/2) marks the holding worker

{queue} is a documentation placeholder for the queue name (EchoMQ.Keys.base/1 is "#{prefix}:#{name}"), not the cluster hash-tag {tag}. The lock manager, the heartbeat that renews the lock, and the stalled sweep that reads expired locks are the dedicated EchoMQ course; this dive teaches the ledger and the ack.

the pattern

The in-flight list is a recovery ledger: a parked job stays there until an explicit ack, so a crashed worker leaves it found and returnable, and the ack runs only when the work is done.

in EchoMQ

EchoMQ.Keys.active/1 is emq:{queue}:active, the ledger; EchoMQ.Keys.lock/2 is emq:{queue}:<id>:lock, the holder mark; an expired lock signals a dead worker for the stalled sweep.

The in-flight list is a recovery ledger, not a buffer; EchoMQ's active/1 list under the lock/2 key is exactly that ledger and its holder mark.

Door · not depth The lock manager that renews each in-flight job's lock on a heartbeat, the stalled sweep (moveStalledJobsToWait-8.lua) that reads expired locks and reclaims their jobs, and the stalled-check coordination across a worker pool are the dedicated EchoMQ course. From here, this dive opens onto E2 · the engine — the as-built lock manager and the worker fetch loop. This dive teaches the ledger and the ack; that course teaches the recovery machinery.

References

The list read as a record, the ack that removes a done job, and the move that parks a job in the ledger.

Sources

Redis — Lists — the list as a record of ordered elements, the in-flight ledger read as a record of work.
Redis — LREM — the ack: remove one occurrence of a job from the in-flight list after the work is done.
Redis — RPOPLPUSH — the move that parks a job in the in-flight ledger atomically.
BullMQ — the queue protocol — the active-list-plus-lock layout EchoMQ ports.

Related in this course

R3.01 · Processing list — the module hub: the in-flight move.
R3.01.1 · List as wait + active — the two named lists.
R3.01.2 · LMOVE / RPOPLPUSH — the atomic move that parks the job in this ledger.
R3 · The reliable queue — at-least-once and stalled recovery, the guarantees this ledger enables.
E2 · the engine — the dedicated EchoMQ course: the lock manager and stalled sweep in depth.