E2.04.2 · The three runtimes · dive 2
The Go gaps
The port speaks the same wire — but it is partial, and a depth course names the gaps rather than smoothing them.
Three things, told straight. The non-atomic add path: where the Elixir
reference runs one atomic addStandardJob-9.lua, the Go port runs two separate commands.
The attemptsMade-vs-atm nuance: Go writes and reads atm on the hash —
only its external JSON tag is attemptsMade, an L4 surface, not a wire bug. And the
cluster validator the Go port does carry: GetClusterSlot +
ValidateHashTags, the CRC16-modulo-16384 rule. Step the add comparison on the right.
Movement I · Core (as-built) ← redis-patterns R3
Add a standard job · one atomic script vs two commands
Gap one — the non-atomic add path
On the Elixir reference, enqueuing a standard job runs one atomic Lua script —
addStandardJob-9.lua, via Scripts.add_standard_job
(scripts.ex line ~435). The job hash is written and the id pushed onto the wait list
in a single round-trip that either fully happens or does not:
# scripts.ex — one atomic script for the whole add execute(conn, :add_standard_job, keys, args) # addStandardJob-9.lua, 9 declared keys
The Go port's add path runs two separate commands: addJobToQueue
writes the hash with HSet (queue_impl.go line ~105), and then
enqueueJob pushes the id with a separate LPush (or
ZAdd for delayed/priority, queue_impl.go line ~137):
// queue_impl.go — addJobToQueue: first command return q.redisClient.HSet(ctx, key, fields).Err() // queue_impl.go — enqueueJob: second, separate command return q.redisClient.LPush(ctx, q.keyBuilder.Wait(), job.ID).Err()
Where the protocol expects one atomic script, the Go port issues two commands. A failure between them can leave
a job hash with no list entry. This is the non-atomic critical path — a real porting gap,
reported, not glossed. (The Go port's fetch path is atomic: the worker pickup runs the
moveToActive Lua script — worker_impl.go line ~112 — so the
gap is on the add side, not the pickup side.)
One atomic script versus two separate commands. The Elixir add either fully happens or does not; the
Go add has a window between HSet and LPush. That is the
non-atomic critical path.
The nuance — attemptsMade vs atm
A common worry is that the Go port writes the wrong field name. It does not. The Go port writes and
reads atm on the Redis hash, exactly as the wire requires:
// queue_impl.go ~102 — WRITE the hash field as "atm" "atm": job.AttemptsMade, // script_runner.go ~160 — READ the hash field "atm" back if attemptsMade, ok := data["atm"]; ok { fmt.Sscanf(attemptsMade, "%d", &job.AttemptsMade) }
Only the Go struct's external JSON tag is attemptsMade
(job.go line 22):
// job.go:22 — the JSON tag is an L4 serialisation surface, NOT the wire AttemptsMade int `json:"attemptsMade"`
That tag is an L4 surface — how a Go program serialises the struct to its own callers — not
the wire. The Redis field round-trips as atm in both directions, so a job written by Go
and read by Elixir (whose EchoMQ.Job.from_redis/4 reads the same compressed fields)
agrees on the field. This was convergently verified twice: it is a serialisation tag, not a wire bug.
The Go port reads and writes atm on the wire; only its JSON output is named
attemptsMade. That is an L4 tag, not a wire difference — the interop holds.
What the Go port does carry — the cluster validator
The Go port carries a real Redis Cluster slot validator the Elixir runtime lacks.
pkg/echomq/cluster.go implements GetClusterSlot and
ValidateHashTags, with RedisClusterSlots = 16384:
// cluster.go — the slot is CRC16 of the {hashtag}, modulo 16384 const RedisClusterSlots = 16384 func GetClusterSlot(key string) int { hashKey := key // extract the {hashtag} content if present crc := CalculateCRC16([]byte(hashKey)) return int(crc) % RedisClusterSlots } func ValidateHashTags(keys []string) (bool, int, []int) { // true iff every key hashes to the SAME slot }
The rule is slot = CRC16(hashtag) % 16384; keys that share a hashtag share a slot,
which is the precondition for running a multi-key Lua script on a Redis Cluster. The checker below applies the
rule to a fixed pair of keys: type a hashtag and read whether two keys built with it land in the same slot. It
demonstrates the rule, not a specific slot number.
ValidateHashTags · same hashtag ⇒ same slot
The cluster validator is a genuine Go-side addition: it confirms a multi-key script's keys share one
slot before running it. The rule is CRC16(hashtag) % 16384 — same hashtag, same
slot.
The Golden Rule — gaps measured against the shared wire
The wire is the same for every runtime — the emq: keyspace and the field names
(including atm) are L1, and a multi-key transition is one atomic Lua
script, L2. The Go port's gaps live above that line, in L3/L4: the add path
runs two commands instead of the atomic script, and the attemptsMade JSON tag is an L4
serialisation choice over the atm field. The cluster validator it adds is also L3 —
the slot rule it checks is the protocol's.
The emq: keyspace and the field names — including atm on
the hash (L1); a multi-key transition is one atomic script (L2) — frozen and
shared.
A partial port — the add path runs two separate commands (the non-atomic gap), the
attemptsMade JSON tag is an L4 serialisation choice over the
atm field, and it adds a CRC16-based cluster slot validator the reference lacks.
The Go port is honestly partial: a non-atomic add path is a real gap, the
attemptsMade/atm mismatch is an L4 tag and not a wire bug, and
the cluster validator is a genuine Go-side addition. A depth course names all three.
Next: with both runtimes covered, how does a job written by one get processed by the other? E2.04.3 · cross-runtime interop walks the round-trip and the v2 trajectory.
The 2.0 fork — EchoMQ leaves the BullMQ wire
The gaps above are measured against the v1 line (frozen at 1.3.0) — the BullMQ wire.
emq.1 ships EchoMQ 2.0, and the Go port must implement
2.0 to stay in the fleet: the v2 declared-keys script set, the emq:{q}:…
keyspace, and the transparent {q} hashtag. The Go port's gap list is its porting
work-list for v2 — the non-atomic add path becomes a v2 atomic script, and the cluster validator it already
carries is exactly the placement check the v2 hashtag design formalises. emq.1's fleet-interop probe tracks the
port's v2 status honestly until it lands.
References
The atomic add and fetch scripts, the cached-script dispatch, and the Redis Cluster slot scheme the validator implements.
Sources
- BullMQ — Documentation — the atomic add and fetch scripts the Go port partially ports.
- Redis — EVALSHA — the cached-script dispatch the atomic fetch path uses.
- Redis — Documentation — Redis Cluster, hash tags, and the CRC16 slot scheme the validator implements.
Related in this course
- E2.04 · The three runtimes & interop — the module hub.
- E2.04.1 · The Elixir architecture — the reference runtime the gaps are measured against.
- E2.04.3 · Cross-runtime interop — the
atmfield round-trip Elixir ↔ Go in full. - redis-patterns R3 · Reliable queues — the atomic-transition pattern the gaps touch.