jonnify·bcs /bcs/cache/single-writer-ring

B4.3 · The Single Writer and the Ring — manuscript chapter 4.3

The single writer applies the stream.

Chapter 4.2 gated coherence's semantics; this chapter changes nothing about them and everything about the engine underneath. Application moves out of the table's owner and onto a bounded ring drained by one applier — the part's fifth law as a data structure. One production module lands (EchoCache.Ring: two atomic sequences, preallocated ETS slots, edge-triggered wakes, batched drains, occupancy as a gauge, drop-on-full as a counted refusal), and the committed record carries the shape in numbers: 1000 items in publish order through 2 batches (largest 801) on 1 wakes; 1005116 items per second on one scheduler; a 500-invalidation storm applied in 25 ms while a fill fired mid-storm completed in 0 ms; and 500 shuffled messages converging on exactly 100 applied and 400 stale.

The chapter is content/bcs4.3.md; the rung behind it is bcs_rung_4_3_check.exs, and its committed transcript closes PASS 6/6 under the header Valkey on 6390 | Elixir 1.14.0 OTP 25 | schedulers 1. Six gates, each derived before it is measured.

The six gates, mapped to the dives. Drawn in transcript order. Select a gate to read its verbatim line and the dive that teaches it.
G1 surface dive 1 G2 order dive 1 G3 occupancy dive 2 G4 full dive 2 G5 storm dive 3 G6 convergence dive 3
readout · Select a gate to read its verbatim line and the dive that teaches it. Six gates, four derive lines, one header; the full record is frozen in §2 below, the dives are mapped in §3.
§1

Why the owner stops applying

content/bcs4.3.md · Why · Decisions

In 4.2 the table's owner did two jobs: it served fills and puts, and it applied every coherence push inline. At 72 microseconds a message that coupling is invisible — until the storm. A halted market segment, a bulk reprice, a reconnect replay: thousands of invalidations arrive in one burst, and an owner that applies inline queues its fills behind them, while an owner that spawns per message buys unbounded process churn for work that is one ETS comparison each. The mature answer has a name and a pedigree: LMAX built a retail exchange whose business logic ran on a single thread — "6 million orders per second on a single thread" — fed by the Disruptor, a bounded ring of preallocated slots with sequence counters. This chapter translates that shape onto the BEAM with the primitives the runtime ships for it, and reads it beside the bus's own park-don't-poll: both replace discovery with arrival, and both wake exactly once per busy period.

BROADCAST ONLY

The ring serves the broadcast lane only

Drop-on-full preserves at-most-once; "routing the job lane's at-least-once delivery through a dropping structure would launder a guarantee away." Two lanes, two engines, one comparison making their races safe.

DROP, COUNTED

Drop, never block, never overwrite

Blocking turns a full ring into owner backpressure and mailbox growth; overwriting loses a message silently and unpredictably. A counted refusal keeps both the lane's contract and the operator's visibility.

ONE PRODUCER

Single producer, structurally

The publish path is lock-free because exactly one process calls it — the owner, where pushes already serialize. The simplicity is bought with a rule, and the rule is stated in the module doc.

CALLER-SIDE

The applier applies caller-side

apply_batch/2 touches public ETS and the spec's counters, never the owner — the whole point of the decoupling — and every interleaving with concurrent fills converges because newer-wins is a comparison.

PERSISTENT_TERM

Runtime in persistent_term

One persistent_term read per publish — the BEAM's cheapest shared-read path, written once at ring start and erased at stop. A brutal kill leaks one entry until the name is reused — "a stated cost of kill-9 truthfulness."

THE GAUGE

Occupancy is the gauge

Tail minus head, readable by anyone at any time. A dashboard that plots this number shows a storm as a hill instead of inferring it from tail latencies.

§2

The proof

content/bcs4.3.md · quoting bcs_rung_4_3_check.out

The full committed record, verbatim — the header, six gates, and four derive lines. The derive lines are part of the record: each measurement is bounded before it is taken, and the committed line then lands inside its own band.

content/echo_data/runtimes/elixir/bcs_rung_4_3_check.out · verbatimfrozen
header: Valkey on 6390 | Elixir 1.14.0 OTP 25 | schedulers 1
G1 surface ok -- the ring's surface is whole -- publish, occupancy, stats, stop, a generic one-batch apply function -- and the declaration tells the truth: the broadcast table carries its ring name and capacity 512 in the directory, the :none table carries nil, and a fresh ring stands at occupancy 0
derive (order): the applier drains everything between head and tail in one pass, so concatenating the batches must reproduce publish order exactly; wakes are edge-triggered on the empty-to-nonempty transition, so 1000 items published into a draining ring should cost a handful of wakes -- well under fifty -- and more than one batch proves the batching is real
G2 order ok -- 1000 items crossed the ring in publish order exactly -- the concatenated batches reproduce the sequence -- through 2 batches (largest 801) on 1 wakes: one message per busy period, not one per item
derive (throughput): a publish is one ETS insert and three atomics operations, near 0.5 to 1 us, and the apply side amortizes to nothing over batches -- so publish cost alone governs, and the end-to-end rate on one scheduler should land between 100,000 and 2,500,000 items per second, floor 80,000; mid-storm occupancy must sit strictly between zero and capacity and drain to exactly zero
G3 occupancy ok -- mid-storm the gauge read 600 of 4096 and drained to exactly 0; priced, the ring moved 100000 items in 99 ms -- 1005116 items per second end to end on one scheduler, inside the derived band, largest batch 200, nothing dropped
derive (full): with capacity 64 and the applier held inside its first apply, exactly 64 publishes are accepted and 136 are refused and counted; releasing the applier drains the 64, and the next publish lands -- the ring under storm refuses, recovers, and keeps serving
G4 full ok -- the bound held its shape: 64 accepted, 136 refused with :dropped and counted -- never blocked, never overwritten -- then the release drained all 64 and publish 201 landed and applied: a storm bends the lane's at-most-once contract no further than the contract already bends
derive (storm): 500 invalidations published on the wire ride push frames at the committed 72 us median into the owner, which only parses and publishes -- application happens on the ring's applier, so a fetch fired mid-storm answers without queueing behind 500 applies; expect the storm applied within two seconds and the mid-storm fill well under 50 ms
G5 storm ok -- 500 invalidations crossed the wire and the ring in 25 ms with nothing dropped; every stormed row left L1, the one name whose writer placed a new value answers px=109.00 from the shared L2, and a fill fired mid-storm completed in 0 ms -- the owner parses and publishes while the applier applies, and neither waits for the other
derive (convergence): for each of 200 names holding version v2, a shuffled stream delivers either v1,v3,v1 or v1,v1 -- whatever the arrival order, a row is dropped if and only if a version newer than v2 appeared, and the per-name verdict counts are invariant under permutation: exactly 100 applied and 400 stale
G6 convergence ok -- 500 shuffled messages converged: the 100 names that saw a newer version lost their rows, the 100 that saw only older versions still answer :hit, and the verdict counters landed exactly on 100 applied and 400 stale -- arrival order changed nothing, because every application is the same comparison
PASS 6/6

The derivation note rides with the record (D-B4.3): "the first band's ceiling undercounted exactly because it priced the apply side that batching had already amortized away." A storm-rated table is declared, not assembled — verbatim from the chapter's How:

{:ok, _} =
  EchoCache.Table.start_link(
    name: :quotes,
    kind: "AST",
    coherence: :broadcast,
    ring_capacity: 4_096,
    ttl_ms: 5_000,
    loader: &PriceFeed.load/1,
    connector: [port: 6390]
  )

The ring lives at runtimes/elixir/lib/echo_cache/ring.ex; the table's growth at lib/echo_cache/table.ex — apply_batch/2 running caller-side against public ETS and the spec's counters, the :broadcast init starting the ring before the subscription that feeds it, the push handler reduced to parse-and-publish, the spec carrying ring and ring_capacity, the terminate stopping the ring with the table.

§3

The dives

three lessons, fixed by the chapter spec

The module hands forward: B4.4 · The Lane That Remembers — the journal writer is the next single-writer candidate; one owner draining ordered work is about to become a habit. And the manuscript plans the referee chapter — B4.5 · The Cache Referee — to hold the comparison set to this module's drills.

§4

References

sources · related

Sources

Related