jonnifyechomq

The Lua layer · dive 03

EVALSHA dispatch

The source crosses the wire once. After that, every call is a digest.

A script's source crosses the wire at most once per connection. EchoMQ.Script.new/2 precomputes each script's SHA1 when the module loads; EchoMQ.Connector.eval/5 runs the script with EVALSHA — sending only the digest, the key count, the keys, and the values. On a cold cache (NOSCRIPT) the connector loads the source once, confirms the returned SHA matches, and re-runs by SHA. Run the dispatch on the right, then read the two code blocks below.

Precompute the SHA Run by SHA, load on NOSCRIPT Verify, never trust References

EVALSHA dispatch · warm cache or cold

step 1 EVALSHA s.sha · KEYS · ARGV warm server runs the script cold NOSCRIPT → SCRIPT LOAD once cold verify SHA · re-run by SHA

Precompute the SHA, at definition

EchoMQ.Script.new/2 is called once per script when the module loads. It hashes the source to a lowercase SHA1 and keeps the name, the source, and the digest together. The source is retained only for the cold-cache load; the digest is what the hot path sends.

# echo_wire — EchoMQ.Script
# A server-side script with its SHA1 precomputed, so the connector can run
# EVALSHA-first with a load-on-NOSCRIPT fallback. Every key a script touches
# is declared in KEYS — the law; ARGV carries values only.
def new(name, source) when is_atom(name) and is_binary(source) do
  # the SHA1 of the source is what EVALSHA addresses; lowercase hex matches
  # the digest SCRIPT LOAD returns, so the two can be compared byte for byte.
  sha = :crypto.hash(:sha, source) |> Base.encode16(case: :lower)
  %__MODULE__{name: name, source: source, sha: sha}
end

So @enqueue = Script.new(:enqueue, "...") is computed once. Every later enqueue/4 reuses the same struct — the SHA is never recomputed on the hot path. The struct carries three fields: the name, the source (for the cold load), and the sha (for the dispatch).

Run by SHA, load on NOSCRIPT

EchoMQ.Connector.eval/5 builds the EVALSHA command from the precomputed SHA, the key count, the keys, and the values. On a NOSCRIPT reply it loads the source once, verifies the SHA, and re-runs; any other server error is mapped to a typed failure.

# echo_wire — EchoMQ.Connector
# EVALSHA-first execution of a declared-keys script. The wire carries the
# 40-char digest, the key count, then KEYS then ARGV — never the source on the
# hot path.
def eval(conn, %Script{} = s, keys, argv, timeout \\ 5_000) do
  parts = ["EVALSHA", s.sha, Integer.to_string(length(keys))] ++ keys ++ argv

  case command(conn, parts, timeout) do
    # cold cache: the server has never seen this script. Load it ONCE, confirm
    # the returned digest equals the precomputed one, then re-run by SHA.
    {:ok, {:error_reply, "NOSCRIPT" <> _}} ->
      GenServer.call(conn, {:bump, 5})   # count the script_load (slot 5)

      case command(conn, ["SCRIPT", "LOAD", s.source], timeout) do
        {:ok, sha} when sha == s.sha -> command(conn, parts, timeout) |> map_script_reply()
        {:ok, other} -> {:error, {:sha_mismatch, other}}
        {:error, _} = err -> err
      end

    # any other server error is a verdict — mapped to a typed {:server, msg}.
    {:ok, {:error_reply, msg}} ->
      {:error, {:server, msg}}

    other ->
      other
  end
end

The hot path is the first command(conn, parts, …): EVALSHA, the digest, the key count, then the keys, then the values. The source appears only in the NOSCRIPT branch, run at most once per script per connection. The connector keeps counters — slot 5 script_loads, slot 6 evalsha_calls — so a healthy connection shows one load per script and many EVALSHA calls after it.

EVALSHA sends a digest, not source. The source crosses once, on a cold cache; every call after that is forty characters and the arguments.

Verify, never trust — two details

Two details carry the dispatch's correctness, and both are in the branch above.

The SHA is verified, not trusted. SCRIPT LOAD returns the server's own digest of the source; the connector accepts the re-run only when that digest equals the precomputed s.sha. A mismatch ({:sha_mismatch, other}) means the source the server hashed is not the source the struct holds — a hard fault, not a silent retry.

A load-and-retry error maps exactly as the first attempt. A script can fail on either attempt; map_script_reply turns the re-run's error reply into the same {:server, msg} shape a first-attempt error gets, so a caller cannot tell a cold cache from a warm one. The connector found this on a cold script cache during conformance.

After SCRIPT LOAD · what the connector does with the returned digest

SCRIPT LOAD returned a digest == s.sha connector does re-run by SHA
the pattern (Redis Patterns Applied)

Load a script once and run it by SHA with EVALSHA, reloading on a NOSCRIPT miss — scripting is how a pattern becomes a protocol. Patterns become protocol and Coordination teach it.

the implementation (echo_wire)

EchoMQ.Script.new/2 precomputes the SHA1; EchoMQ.Connector.eval/5 runs it EVALSHA-first, loads the source once on a NOSCRIPT miss, verifies the returned digest, and maps a re-run error exactly as a first-attempt one.

The dispatch trusts nothing it can verify: it confirms the loaded digest, and it makes a cold cache indistinguishable from a warm one.

Recap — the digest, not the source

EVALSHA dispatch sends a digest, not source. EchoMQ.Script.new/2 precomputes the SHA1; EchoMQ.Connector.eval/5 runs the script by SHA, loads the source once on a NOSCRIPT miss, verifies the returned digest, and maps a re-run error exactly as a first-attempt error. The scripts are the protocol; this is how they run. That closes the Lua layer — the next module reads what holds the wire together.

Next module: Immutability & branded ids — why the keys and the Lua are held fixed, the 14-byte branded-id gate, and the version fence.

References

Where the EVALSHA dispatch is grounded, and the load path it falls back to.

Sources

Related in this course