jonnify·redis /redis-patterns/coordination/redlock/single-instance-enough

R2.03.3 · The Redlock algorithm — dive 3

When single-instance is enough.

Redlock works. The question this dive resolves is whether a workload needs what it buys — because the five-instance cost is real.

Grounding (contrast): echo/apps/echo_mq/jobs.ex:126 · consumer.ex:57,93 — EchoMQ deliberately chose the single-Valkey lease: @claim against one server clock, the attempts fence, the Consumer’s :lease_ms default 30_000, and Jobs.reap.

The decision matrix — set the two axes. Choose the consequence of a lost lock and the budget; the matrix names the right tool, and the readout explains it.
consequence of a lost lock
infrastructure budget
budget: one Valkey budget: five masters lost lock: recoverable lost lock: catastrophic recoverable · one Valkey single-instance lease (EchoMQ’s choice) recoverable · five masters single-instance suffices Redlock over-provisions catastrophic · one Valkey consensus (etcd / ZK) a lock is not enough catastrophic · five masters Redlock + fencing token consensus if no fence selected: recoverable · one Valkey

Static verdicts — recoverable + one Valkey → single-instance lease · recoverable + five → single-instance suffices, Redlock over-provisions · catastrophic + one Valkey → consensus · catastrophic + five → Redlock + fence.

§1

When to use Redlock

redlock.md.txt · when to use

Redlock is the right tool for a specific shape of problem: coordination across machines where a lost lock is costly enough to justify fault tolerance, but not so catastrophic that only a consensus system will do. The source names four fits and four signals to reach elsewhere.

Redlock is appropriate for coordinating access to an external resource that has no concurrency control of its own; for preventing duplicate job execution in a distributed task queue; for distributed rate limiting where approximate correctness is acceptable; and for leader election among application instances. Each shares a trait: the work is coordination, the consequence of a brief overlap is tolerable or recoverable, and the operational cost of five masters is worth paying for fault tolerance a single instance cannot give.

Reach for an alternative when the shape is different. Use a consensus system — etcd or ZooKeeper — when absolute correctness is required and a lock violation cannot be tolerated. Use fencing at a linearizable resource when the storage layer can enforce a monotonic token, a stronger guarantee than any lock. Stay with single-instance Redis when its reliability is sufficient. And avoid Redlock when the operational complexity of five independent masters is not justified by the failure it prevents. That last line is the hinge: Redlock’s price is not the algorithm — it is the infrastructure.

Cost vs guarantee — step through the workloads. Select a workload; the bar is the guarantee the work demands, and the readout names the right tool.
■ single-instance sufficient ■ stronger guarantee required job queue money transfer leader election rate limit

Static verdicts — job queue / rate limit → single-instance sufficient (short) · leader election → Redlock-class fits (medium) · money transfer → a fence or consensus (long, corrupts a balance).

The bar is the guarantee the work demands, not the lock on offer. Three of the four workloads are short; the money transfer is long, because a lost lock corrupts a balance and only a fence or a consensus system holds.

§2

The decision

redlock.md.txt · the decision

The whole module collapses to one decision, and it is about the consequence of a lost lock, not the lock itself. A lock can always be lost — a failover, a clock jump, a process pause past the lease. What separates the right answer is what happens next.

If a lost lock causes catastrophic corruption — two writers mutate a balance, a transfer is applied twice with no record, an external system is driven into an inconsistent state it cannot recover from — then the lock is a correctness boundary, and a lock that can fail under timing assumptions is the wrong boundary. The answer is a consensus system that does not rest on bounded clocks, or a resource that enforces a fencing token, so a stale holder is rejected where the data changes.

If a lost lock causes a recoverable retry — the same work runs twice, and running it twice is safe because the operation is idempotent and a recovery path reclaims the orphaned unit — then the lock is an optimization, not a correctness boundary. Here a single-instance lease is sufficient, and the five-master cost buys a guarantee the workload does not need. A job queue lands squarely on the recoverable side, and that is the contrast EchoMQ makes concrete.

The axis is the consequence of a lost lock: catastrophic corruption needs more than a lock; a recoverable retry needs less. A job queue reads the recoverable cell.

§3

The summary table

redlock.md.txt · the trade

The source closes on a four-row comparison. It is the trade in one frame: Redlock buys fault tolerance with complexity, and a single instance keeps it simple at the cost of surviving a node failure.

AspectSingle instanceRedlock
Fault toleranceNone — a node failure loses the lockTolerates a minority of node failures
ComplexityLow — one Redis, one SET … NX PXHigher — five independent masters, the acquire-and-time-check protocol
ConsistencyLost on failover (the master-replica race)Maintained while a majority survives
Use caseNon-critical coordination, or recoverable workImportant distributed coordination

The trade has no universal winner. Redlock buys fault tolerance with five masters; a single instance keeps one and accepts that a failover drops the lock. The workload picks the axis.

§4

The contrast, resolved — EchoMQ keeps one Valkey

echo_mq/jobs.ex:126 · consumer.ex:57,93

EchoMQ takes the single-instance side, and the choice is deliberate, not a compromise. A job queue does not need absolute mutual exclusion; it needs one active worker per job in the common case and a safe recovery when that breaks. The lock is the claim lease on one Valkey. The @claim script (echo/apps/echo_mq/lib/echo_mq/jobs.ex:126, run by Jobs.claim/3 at jobs.ex:283) reads one server clock and stamps the lease — no majority of N and no second master.

# @claim (jobs.ex:126) — the single-Valkey lease, on one server clock, with the attempts fence
local att = redis.call('HINCRBY', jk, 'attempts', 1)   # the monotone fence, returned to the worker
redis.call('HSET', jk, 'state', 'active')
local t = redis.call('TIME')                           # one clock — no five masters, no vote
local now = t[1] * 1000 + math.floor(t[2] / 1000)
redis.call('ZADD', KEYS[2], now + tonumber(ARGV[2]), id)

The reason this is safe is the recovery path, not the lock. The EchoMQ.Consumer holds the lease for :lease_ms (default 30_000, echo/apps/echo_mq/lib/echo_mq/consumer.ex:57); on each beat it calls Jobs.reap (consumer.ex:93), which returns every expired lease in active to pending on the same server clock. The next worker re-claims the job and increments attempts; when the original worker resumes, @complete/@retry refuse its stale token with EMQSTALE … token mismatch. Because the handler is idempotent, the re-run produces the same result. A lost lease becomes a double-run the queue recovers from, not two writers corrupting a resource — exactly the move Redlock’s critics prescribe, applied where the data changes instead of at the lock.

Notes on ValkeyTIME reads the server’s own clock, and a Lua script runs atomically, so the lease, the fence, and the reclaim each land as one transition — one Valkey, no majority — valkey.io/topics/programmability.

the pattern

Redlock buys fault tolerance with five independent masters — the right call when a lost lock corrupts data and no cheaper guarantee holds the line.

its EchoMQ application

@claim holds the lease on one Valkey against one server clock, with attempts (HINCRBY) the fence refused stale with EMQSTALE; the EchoMQ.Consumer holds it for :lease_ms (30_000) and Jobs.reap reclaims a lapsed lease as a safe idempotent re-claim — so the queue keeps one Valkey instead of five.

Reach for Redlock when a lost lock corrupts; a queue makes the work idempotent, reads one server clock, and keeps one Valkey. The five-master cost buys a guarantee a recoverable workload does not need.

a door, not a depth · the EchoMQ course

The single-Valkey lease, the version fence

EchoMQ does not implement Redlock — this dive cites the single-Valkey lease as the road not taken: @claim on one Valkey, the attempts fence refused stale with EMQSTALE, the Consumer’s :lease_ms, and Jobs.reap reclaiming a lapsed lease. The committed keyspace record:

content/bcs3.1.md · the v2 keyspace, the version fencefrozen
emq:{orders}:pending | emq:{orders}:job:ORD0NgWEfAEJfs | {emq}:version | {emq}:locks   ← 17 bytes before the payload
GET {emq}:version              echomq:2.0.0   ← the version fence, through the fenced connector

The worker-side lock plane and the full v2 script bundle are the subject of the dedicated EchoMQ course; where a cache fact applies, open The Branded Component System.

§5

Recap — the contrast resolved

back → R2.03

Redlock is a real tool with a real price: five independent masters, no replication, the acquire-and-time-check protocol. The source names where it fits — external resources, duplicate-job prevention, approximate rate limiting, leader election — and where to reach past it: a consensus system when correctness is absolute, a fenced resource when the storage layer supports a token, a single instance when its reliability is sufficient. The whole module reduces to one axis: a lost lock that causes catastrophic corruption needs more than a lock; a lost lock that causes a recoverable retry needs less. EchoMQ reads the recoverable cell — @claim on one Valkey, the attempts fence, and Jobs.reap reclaiming a lapsed lease, with an idempotent handler that makes a re-run safe — and keeps one Valkey.

§6

References

sources · related

Sources

Related in this course