B4.1 · dive 2 — one fill per herd
Two hundred misses, one load.
Misses route through the table's owner, and concurrent misses on one key coalesce onto a single flight — the first caller's flight checks L2, falls through to the loader, writes both layers, and the owner replies to every waiter with the one answer. The drill is the gate: 200 concurrent cold readers, loader runs 1, coalesced waiters 199, every reader holding the one answer. The hit path that makes the cache worth having is priced on the same record: 1311621 hit reads per second (762 ns each) against 31 us per L2 GET on the same wire.
Source: content/bcs4.1.md, quoting bcs_rung_4_1_check.out; the module is
committed at runtimes/elixir/lib/echo_cache/table.ex.
The transcript
bcs_rung_4_1_check.out · verbatim · this dive reads the herd and speed lines… derive (herd): 200 concurrent cold readers without single-flight run 200 loads; the law demands the misses coalesce onto one flight -- expect loader runs 1 and 199 coalesced waiters E3 herd ok -- the thundering herd survived with one fill: 200 concurrent cold readers, loader runs 1, coalesced waiters 199, every reader holding the one answer derive (speed): a hit is a caller-side lookup on a public read-concurrency set plus the kind gate and a counter bump -- expect 250,000 to 1,500,000 hit reads per second on this core; an L2 GET pays a loopback round trip, and Appendix A committed 29,456 sequential round trips per second, near 34 us each -- expect the L1 hit at least 10 times cheaper than the wire E4 speed ok -- measured: 1311621 hit reads per second (762 ns each) against 31 us per L2 GET on the same wire -- the L1 hit is 40 times cheaper than the round trip it replaces, inside the derived band … PASS 6/6
The elided lines hold the header, E1–E2, and the jitter and bound drills — dive 1 and dive 3 read them, and the module hub freezes the record whole. The derive lines are part of the record: each measurement is bounded before it is taken, and the committed line then lands inside its own band.
The flight
content/bcs4.1.md · What · DecisionsThe first caller's flight checks L2, falls through to the loader, writes both layers,
and the owner replies to every waiter with the one answer. The pattern has a name in the Go world —
singleflight, "a duplicate function call suppression mechanism" — and the Go port's fill discipline
already has its idiom: golang.org/x/sync/singleflight is one-fill-per-herd as a library.
The decision: flights are processes, not owner code. A slow loader blocks its own flight, never the owner — other keys keep filling, puts keep landing. A crashed flight is a monitored event: every waiter gets a typed error, nobody hangs. And loader errors are not cached — a broken upstream surfaces immediately instead of being memorized; negative caching is a deliberate omission, revisitable under its own gate.
Two hundred simultaneous cold readers at the open are the normal case in this domain, and the gate says they cost one load. The manuscript plans the referee chapter — B4.5 · The Cache Referee — to hold the comparison set to this drill.
The priced pair
content/bcs4.1.md · Hits at ETS speed · BoundariesA hit is a caller-side lookup against a public, read-concurrency ETS table, plus the
kind gate and one atomic counter bump. The read path never enters the owner's process — the owner is
consulted only when there is owner's work to do; that is the whole architecture. Derived first, then
measured: 1311621 hit reads per second (762 ns each) against 31 us per L2 GET on the same wire --
the L1 hit is 40 times cheaper than the round trip it replaces, inside the derived band.
This is what makes 762 ns a hit — always read beside its pair, the 31 us the wire
charges — and it is why stats ride :counters instead of GenServer.call. The
boundary travels with the number: one core, loopback, this container — the ratios travel, the absolutes
describe this machine.
Static record — the pair and its derivation: 1311621 hit reads per second
(762 ns each) · 31 us per L2 GET · the derive line's band
250,000 to 1,500,000 hit reads per second on this core · Appendix A's floor
29,456 sequential round trips per second, near 34 us each.
References
sources · relatedSources
- Go x/sync — singleflightduplicate function call suppression: the named prior art for one fill per herd, and the Go port's idiom for it
- Erlang/OTP — etspublic read_concurrency tables: the caller-side hit path
Related
- B4.1 — Cache-Aside at ETS Speedthe module hub; the full rung in context
- B4 — EchoCachethe chapter landing
- B3 — The Busthe wire whose loopback floor prices the comparison
- B2.2 — Property Storesthe stores behind the loader
- Redis Patterns Appliedthe caching substrate patterns
- EchoMQ — the coursethe bus protocol in rung-level depth