jonnify·bcs /bcs/ideas/id-system/the-streams-horizon

B1.3 · dive 4 of 4 — the entry-id injection

The id carries its window.

Stream entry ids are millisecond-sequence pairs, range-queryable by time because the id carries it. The injection unix_ms(snow) dash low-22-bits(snow) is order-preserving — carrying the contract into the entry id costs nothing and buys exact replay windows.

Measured on 200,000 entries per stream against Valkey; the figures are verbatim from streams_bench.out. EchoMQ 3.0 plans Streams with PubSub — this is the groundwork that plan stands on.

The injection. The snowflake's bit fields split into the entry id's two halves. Select a committed vector; the mapping is computed live by pure functions.
snowflake — ts(41) << 22 | node(10) << 12 | seq(12) ts · 41 bits node · 10 seq · 12 + epoch 1704067200000 the low 22 bits together unix_ms(snow) – low-22-bits(snow) the stream entry id — millisecond, dash, sequence "1781000000010-28672"
readout · The fixed dataset is the two committed vectors of content/vectors.json and the bench's own first id. Select one; the entry id is computed live — unix_ms(snow), a dash, the low 22 bits.
§1

The entry id

content/bcs1.3.md · The streams horizon

A stream entry id is a millisecond-sequence pair, range-queryable by time because the id carries it, under the same monotonic top-id law the contract's minter obeys. The injection unix_ms(snow) dash low-22-bits(snow) is therefore order-preserving: the snowflake's high bits are the milliseconds, and the low 22 bits — node and sequence — break ties within the millisecond the same way on both sides of the mapping.

§2

Measured

bench/valkey-id/streams_bench.out
content/echo_data/bench/valkey-id/streams_bench.out · verbatimfrozen
# streams: Valkey, 200,000 entries, one 8B field/value pair each
s_auto  entries=200000 bytes=4174928 per-entry=20
s_brd   entries=200000 bytes=4161888 per-entry=20
window [+10ms, +20ms) via branded-derived ids: 40960 entries (expected 40960)
first id in window: "1781000000010-28672"  (low 22 bits = node 7 << 12 | seq 0 = 28672)

Explicit branded-derived ids cost the same 20 bytes per entry as XADD * auto-ids, and a ten-millisecond window addressed purely by id arithmetic returned its predicted 40960 entries. Carrying the contract into the entry id costs nothing.

§3

The window arithmetic

streams_bench.out · vectors per content/vectors.json

What the injection buys: exact replay windows addressed by the same min_for arithmetic every runtime already implements — a half-open pair of instants becomes a half-open pair of entry ids, with no second mapping to maintain. The buttons verify the bench's facts live:

check · Select a check. Each one recomputes a committed figure with a pure function and compares the result against the frozen line above.
§4

The recipes

content/bcs1.3.md · How

The fleet guarantee is that two runtimes shaping the same key produce the same bytes — which the canon's conformance suites already prove for the branded payload, leaving only the envelope to discipline:

# Elixir — the connector side
job_key   = "emq:{" <> queue <> "}:job:" <> EchoData.Snowflake.next_branded("ORD")
stream_id = "#{EchoData.Snowflake.unix_ms(snow)}-#{Bitwise.band(snow, 0x3FFFFF)}"
// Go — the consumer side
jobKey   := "emq:{" + queue + "}:job:" + brandedid.MustEncode("ORD", snow)
streamID := fmt.Sprintf("%d-%d", brandedid.UnixMs(snow), snow&0x3FFFFF)

Same grammar, same accessors-by-contract, byte-identical keys — so the hashtag's slot decision agrees across the fleet, and an ORD replay window computes the same range on either side of the bus. The injection is the one stream-id scheme (INV-K4 in the chapter's spec of record); a parallel mapping would be the second clock in new clothing.

Where the entry id meets the bus protocol itself — queues, groups, consumers — that is the EchoMQ course's depth, through the door below.

§5

References

sources · related

Sources

Related