R1.07.1 · Caching workshop — stage 1

Cache the catalog.

Put a cache-aside read in front of one real consumer surface — the catalog of tradeable instruments the Exchange Platform’s gateway checks on every order. One read path, an L1 ETS hit or a single-flight fill.

Grounding: echo_cache/table.ex (fetch/3, launch_flight) · echo_cache/keyspace.ex · exchange/gateway.ex (parse_instrument) · content/bcs4.1.md (the gate, Valkey).

Three sources of one answer. Select a source to read how fetch/3 resolves it and what it costs.

readout · Select a source. A read resolves from L1, then L2, then the loader — in that order, returning the source it answered from.

Static verdicts — L1 hit → :ets.lookup, no owner call · L2 hit → GET frames the version off the front · fill → loader, then SET … PX, both layers.

§1

Two reads, one declared cache

echo_cache/table.ex · keyspace.ex

The catalog read resolves an instrument by its branded id. EchoCache.Table.fetch/3 is the whole surface, and it answers from one of three sources, in order: an L1 hit in the caller’s own process, an L2 hit on the shared Valkey, or a fill through the declared loader. The return is {:ok, value, source} with source :hit | :l2 | :fill — the source tag is observability, not ceremony.

The key form is the cache’s own: ecc:{instruments}:<id>, built by EchoCache.Keyspace.key/2 as "ecc:{" <> table <> "}:" <> id. The table name is hash-tagged inside the braces so every key of one cache lands in one cluster slot the day clustering arrives — a fresh prefix beside the bus’s emq:, never inside it. The id in the key’s value position is checked for shape before any key is composed; a malformed name never reaches the wire.

The read path never enters the table’s owner process on a hit. A hit is a caller-side :ets.lookup against a public, read-concurrent table, plus the kind gate and one atomic counter bump. The owner is consulted only on a miss — and that is the second law: one fill per herd, the next stage’s territory by construction.

fetch/3: The read. Returns {:ok, value, :hit | :l2 | :fill}, {:error, :kind} for a wrong-namespace id, or the loader’s error.
ecc:{instruments}:<id>: The L2 key form (Keyspace.key/2), the table name hash-tagged for one cluster slot.
the kind gate: byte_size(id)==14 and binary_part(id,0,3)==kind and BrandedId.valid?(id) — refused at the door, before either layer.
L1 ETS: The hit path: a public, read-concurrent table read in the caller’s process at memory speed.

Notes on ValkeyThe L2 row is written with SET … PX — value and expiry in one atomic command — so the second layer expires itself even if every node forgets; the committed gate reads PTTL 300 ms of 300 — valkey.io/commands/set.

§2

Lazy fills, read by read

echo_cache/table.ex:391 · bcs4.1

A cold cache fills itself from real traffic. The first request for an instrument misses, and fetch/3 routes through the owner to a single-flight fill — launch_flight/2 runs GET ecc:{instruments}:<id>; on {:ok, nil} it calls the declared loader, then writes both layers with SET … PX. Every request after it — until the row expires or is dropped — hits L1 in its own process. The cost of a cold start is a number that falls as instruments warm: the loader runs once per fill.

Run a short trace of catalog reads below against a cache that starts empty. The first touch of each instrument is a miss and a fill; a repeat touch within the window is an L1 hit. The readout counts the loader runs the trace caused and the round trips, so the cold-start cost is a number you can see fall as instruments warm.

Run a read trace. Cold misses warm into L1 hits; the readout counts the loader runs.

L1 TTL (reads) 8

A cold cache warms read by read: the first touch of each instrument misses and fills with SET … PX; every later touch hits L1 at ETS speed. The L1 hit is 40 times cheaper than the L2 round trip it replaces.

§3

The catalog read path on EchoCache

exchange/gateway.ex · echo_cache/table.ex

Take one read: the gateway resolves an instrument to validate an order. The call is EchoCache.Table.fetch/3 against the instrument table. On a hit it returns from L1 ETS; on a miss the owner launches a single flight that checks L2, falls through to the loader, writes both layers, and replies to the caller — and to any waiter that coalesced behind it — with the one answer. The gateway receives the instrument or, for one the venue does not list, {:error, :unknown_instrument} from parse_instrument/1; the hit-or-miss branch lives in the cache.

# The catalog read path — one fetch/3, three sources; the source tag is observability.
{:ok, instrument, source} = EchoCache.Table.fetch(:instruments, id)
# source :: :hit | :l2 | :fill

# Inside the owner, on a miss — launch_flight/2 (echo_cache/table.ex:391):
case Connector.command(conn, ["GET", l2]) do   # l2 = "ecc:{instruments}:" <> id
  {:ok, nil} ->                                  # miss: run the loader, then fill both layers
    {:ok, value, version} = run_loader(loader, id)
    {:ok, "OK"} = Connector.command(conn, ["SET", l2, version <> value, "PX", ttl_ms])
    {:fill, value, version}

  {:ok, <<version::binary-14, value::binary>>} -> # L2 hit: frame the version off the front
    {:l2, value, version}
end

The L2 value is framed — a 14-byte branded version prefix in front of the bytes — so a later read can compare the version newer-wins. Both layers are written under one TTL, on the server’s clock for L2 and a jittered monotonic clock for L1. How the loader reads and shapes the source is /elixir, not repeated here.

the application · the read at ETS speed, gated

EchoCache.Table against live Valkey

The hit path and its cost are measured, not assumed — the committed EchoCache gate against the live engine:

content/bcs4.1.md · the EchoCache gate, against live Valkeyfrozen

L1 ETS hit                    1311621 reads/s   ← 762 ns each, caller-side :ets.lookup
L2 GET                        31 us          ← the L1 hit is 40 times cheaper
L2 row written with           SET … PX       ← PTTL 300 ms of 300
the read path                 :hit | :l2 | :fill

the pattern

Cache-aside read: check the cache, and on a miss run the loader and fill. The cache fills lazily, one requested key at a time, and a hit never pays the round trip.

its EchoCache application

EchoCache.Table.fetch/3 runs that read path over the instrument catalog the gateway checks — :hit | :l2 | :fill — and a wrong-namespace id is refused at the door.

One read path serves the whole catalog: one fetch/3, three sources, and a cache that warms from real traffic.

§4

Recap — the base layer is in place

next → R1.07.2

The instrument catalog now reads through EchoCache. Every lookup is one fetch/3: an L1 ETS hit, an L2 hit, or a single-flight fill that runs the loader and writes both layers with SET … PX. The cache warms lazily and every L2 fill carries a TTL. What it does not yet do is react to a change — a re-listed instrument keeps serving its old cached copy until the TTL expires. The next dive drops the L1 row on a change and resolves newer-wins.

next · R1.07.2

References

sources · related

Sources

Valkey — SET — SET … PX sets a value and its TTL in one atomic command; the L2 fill of the single-flight load.
Valkey — Topics — the engine the EchoCache gate is measured against, Valkey.
Redis — GET — the read that opens the L2 path; returns the framed value or nil on a miss.
Sanfilippo, S. — antirez weblog — the Redis creator on expiry and treating cached values as a disposable copy.

Related in this course

R1.07 · Caching workshopthe workshop hub
R1.07.2 · Keep it consistentthe next stage
R1.01 · Cache-asidethe base pattern
R1 · Cachingthe chapter
/bcs · cache-asidethe EchoCache manuscript chapter
/elixirthe OTP craft behind the loader