B8.2 · dive 3 — the double-entry ledger
The stream for unsettled, the ledger for settled.
The platform keeps two records and decides, in the open, which holds which truth. The
stream — the journal's log of fills — is the source of truth for unsettled state; the
Postgres ledger is the regulated record of settled positions. Settled money posts double-entry,
every posting one Ecto.Multi. The compaction that keeps the log small is as-built
EchoCache.Journal behavior, quoted from its record; the Trading.Ledger that posts
the settled side is a PROPOSED design object.
Source: compact(j) and the writer's price are real Elixir at
echo/apps/echo_cache/lib/echo_cache/journal.ex, recorded in bcs_rung_4_4_check.out
(PASS 6/6). Trading.Ledger and Exchange.Projection are PROPOSED
(docs/trading/) — taught as design, never as a measured number.
Ecto.Multi. The readout checks the postings sum to zero, and that a
malformed posting aborts the whole multi. The shape is design; no number is claimed.Ecto.Multi — a debit and a credit that must sum to zero. Post a balanced
fill, or attempt an unbalanced one and watch the whole multi abort. This is PROPOSED design; no
latency or posting number is claimed.Compaction is coverage — the outbox empties, the memory does not
bcs_rung_4_4_check.out · H5 · verbatimThe same coverage predicate that drives replay retires the outbox: an intent is retired when its name carries an applied version at least as new — coverage, not acknowledgment. After the loss drill closed at 50 of 50, all 50 intents are deletable, the applied memory keeps all 50 names, replay finds nothing, and a reopen still remembers. The committed gate:
derive (compaction): an intent is retired when its name carries an applied version at least as new -- coverage, not acknowledgment -- so after H4 all 50 intents are deletable, the applied memory keeps all 50 names, replay finds nothing, and a reopen still remembers H5 compaction ok -- all 50 intents retired by coverage in one pass, the applied memory kept its 50 names, replay over the compacted journal found nothing to do, and a fresh open of the same file still answers the last word -- the outbox empties, the memory does not PASS 6/6
compact(j) is the retention verb, idempotent and safe to run on a schedule; the
outbox stays small, the last word per name persists. Intents are small, coverage is one SQL pass, and an
outbox kept under a few thousand rows replays in milliseconds.
The price of memory
bcs_rung_4_4_check.out · H6 · verbatimThe journal's durability is not free, and the record prices it exactly. On prepared-once
statements — the single writer's privilege, with bind resetting the statement — the writer's pair is two WAL
commits and one cached rowid read at synchronous=NORMAL. The committed gate:
derive (price): on prepared-once statements -- the single writer's privilege, with bind resetting the statement -- the writer's pair is two WAL commits and one cached rowid read at synchronous=NORMAL, so expect between 20 and 250 us on this disk; the remembered lane's end-to-end median should land between 200 us and 2 ms against the bare lane's committed 148 us: dearer, bounded, and the chapter's reason the journal is declared per group rather than assumed H6 price ok -- the memory's price on this disk: 143 us per record-and-mark pair at the writer's edge, and a remembered lane end-to-end median of 524 us against the bare lane's committed 148 us -- 3.5 times the latency buys an outbox, a last word per name, and a replay that survives the bus PASS 6/6
148 µs, the remembered lane's 524 µs median, and the 143 µs record-and-mark pair — drawn to scale, with the 3.5× multiple labelled. Every value is the H6 record's own; no platform number is added.3.5 times the latency — and 143 µs per
record-and-mark pair at the writer's edge. The two latency numbers travel together.The priced pair is 524 µs (the remembered lane's end-to-end median) against the
bare lane's committed 148 µs — the two numbers travel together — and 143 µs is the
record-and-mark pair at the writer's edge. At synchronous=NORMAL the WAL issues no per-commit
sync — the checkpoint is the sole I/O barrier — which is the journal's speed and its stated power-loss trade.
The numbers carry their header: one scheduler, this container's disk, a 29-byte payload; the 3.5 multiple
travels better than the 143.
The regulated ledger — PROPOSED
docs/trading/trading.specs.md · design voiceThe settled side is the design's Trading.Ledger (status PROPOSED).
Double-entry in Postgres: branded id columns under the SQL canon with the domain's reject table as the floor,
every posting one Ecto.Multi — a single all-or-nothing transaction, so a fill either posts both
entries or neither. The directional decision is named in the spec and revisitable only by editing the spec:
the stream (the journal, then milestone-B stream lanes) is the source of truth for unsettled state;
the Postgres ledger is the regulated record of settled positions.
Exchange.Projection (also PROPOSED) is the idempotent log consumer that
folds the log into Tables — the read path the platform serves positions and balances from. Both modules are
taught as design; neither has a rung yet, and so neither claims a latency, throughput, or posting number. The
first such number lands when the first rung's harness runs (docs/trading/trading.roadmap.md,
TRD.3 and TRD.5).
Why the split, weighed
docs/trading/trading.specs.md · the design's reasoningThe two-store decision is argued, not assumed.
| Option | What it buys | Why it is not the whole answer |
|---|---|---|
| one Postgres ledger | regulated and queryable for everything, settled or not | every unsettled tick becomes a synchronous SQL write on the hot path, where the fold of an in-memory log answers for free |
| one stream only | replayable and fast, no relational dependency | a regulated record of settled money wants the SQL canon's constraints, the reject table, and the audit a relational ledger gives |
| the split (chosen) | unsettled where replay is cheap, settled where regulation is required | the boundary is named in the spec, decided in the stream's favor for unsettled and Postgres for settled — revisitable only by editing the spec |
The design splits the difference along the settlement boundary. The Postgres canon the ledger posts to is the substrate of /redis-patterns and /elixir; the stream lanes the unsettled log moves to are /echomq. The module returns to the capstone at B8.2 · The Log and the Ledger; the strategies that emit intents over this same log are the subject of B8.3 · Strategies as Deciders, a PROPOSED forward module.
References
sources · relatedSources
- Richardson — Pattern: Transactional outboxthe outbox the journal compacts; coverage retires it without acknowledgment
- SQLite — Write-Ahead Loggingat synchronous NORMAL the checkpoint is the sole sync barrier — the journal's speed and the price this dive quotes
- Litestream — How it worksthe off-box replica that completes the journal's durability story beside the box
Related
- B8.2 — The Log and the Ledgerthe module hub; the full journal record in context
- B4.4 — The Lane That Remembersthe journal and the price quoted here, in its home chapter
- B4 — EchoCachethe chapter the journal lives in
- B3 — The Busthe work lanes settlement jobs drain
- Functional Programming in Elixirthe umbrella and the SQL canon
- Redis Patterns Appliedthe durable-memory substrate