Processing list

A naive queue runs four steps. A producer pushes a message with LPUSH queue "message". A consumer takes it with RPOP queue, which removes it from Redis. The consumer then crashes during processing. The message is gone permanently — not in the queue, not finished, with nothing to recover it from.

The flaw is the RPOP. It removes the message before any work has run, so between the pop and the finish there is a window in which the message is in neither place. A deploy, an OOM kill, or a power loss in that window drops the message. The toggle below runs the same pickup two ways — a plain RPOP that loses the message on a crash, and the atomic move that does not — over a fixed job set, and reports which list holds the job and whether it survives.

Instead of removing the message, atomically move it to a "processing" list. The message stays in Redis until it is explicitly deleted after successful processing. The transfer is one command — RPOPLPUSH or the modern LMOVE — so there is no instant at which the message is in neither list.

A crashed worker therefore leaves the message parked in the processing list, where it can be found and returned. The naive form removes the message and bets the worker survives; the atomic form moves the message and keeps it recoverable whether the worker survives or not. That single change — move, do not remove — is the whole pattern.

The consumer uses LMOVE (or the older RPOPLPUSH) to atomically transfer a message from the main queue to a processing queue. The one command removes the message from the right of work_queue, adds it to the left of processing:worker1, and does both atomically — no message loss is possible.

The older RPOPLPUSH source dest is the same move in fixed directions: RPOPLPUSH src dst is exactly LMOVE src dst RIGHT LEFT. After successful processing, remove the message from the processing queue with LREM processing:worker1 1 "message". For consumption without polling, the blocking form BLMOVE work_queue processing:worker1 RIGHT LEFT 30 waits up to 30 seconds for a message rather than returning immediately on an empty queue. The second dive walks RPOPLPUSH vs LMOVE and the blocking variant.

A message moves through three steps. Dequeue: LMOVE work_queue processing:worker1 RIGHT LEFT. Process: the application handles the message. Acknowledge: LREM processing:worker1 1 "message".

If the worker crashes between steps 1 and 3, the message remains in processing:worker1. That parked message is the seam reliability rests on: it is still in Redis, attributable to a worker, and returnable to the main queue for another attempt. Delivery is therefore at-least-once — a message is delivered one or more times, never zero — which the next module makes the consumer idempotent for, and the module after reclaims a stalled message with. This module stops at the move; the parked message is the third dive.

Each worker should have its own processing queue — processing:worker1, processing:worker2. A per-worker list lets a recovery monitor attribute a parked message to the worker that holds it, and return it cleanly. The processing list is not only a buffer; it is the recovery ledger, a list of exactly the messages currently in flight per worker.

Reading a worker's list answers what work that worker holds right now, without any extra bookkeeping. When a worker dies, a monitor reads its list, finds the parked messages, and returns them to the main queue. The third dive reads the in-flight list as that ledger and pairs it with the lock that marks the holding worker.

Each dive takes one layer of the move, in source order: the list as a queue underneath everything, the atomic transfer command that closes the gap, and the in-flight list read as the recovery ledger. Read them in order — the primitive first, the command second, the ledger third.

The Redis commands behind the atomic transfer — the list primitive, the move, its modern successor, and the ack — and where the worker path that runs them is taught.