The stateless queue

There is no in-memory queue object that holds jobs — Redis holds all the state.

EchoMQ.Queue.add("emails", "send", %{...}, connection: :redis) works with a string name and a connection — no stateful process — because Redis holds the queue state: the wait LIST, the delayed and prioritized ZSETs, and the job hashes. Toggle the queue on the right between "a stateful object holds the jobs" and "Redis holds the state" to read what survives a process restart.

Two clauses, one stateless path The add runs the matching script What survives a restart The keys are the queue References

Who holds the jobs? · restart the process

Two clauses, one stateless path

EchoMQ.Queue.add(queue, name, data, opts \\ []) (arity 4) has two verified clauses. The atom/pid clause is a convenience wrapper around a named or pid Queue process; the binary clause is the real shape of the queue — a name, a connection, and nothing held in memory.

# EchoMQ.Queue.add/4 — two clauses (lib/echomq/queue.ex, verified)

# convenience: a named or pid Queue process
def add(queue, name, data, opts) when is_atom(queue) or is_pid(queue) do
  GenServer.call(queue, {:add, name, data, opts})
end

# the stateless path: a string name + a connection is the whole queue
def add(queue, name, data, opts) when is_binary(queue) do
  conn   = Keyword.fetch!(opts, :connection)
  prefix = opts[:prefix] || "bull"
  ctx    = Keys.new(queue, prefix: prefix)
  job    = Job.new(queue, name, data, opts)
  add_job(conn, ctx, job)
end

The binary clause holds no jobs. It reads the :connection, builds a key context with Keys.new/2, builds the Job with Job.new/4 (E2.02.1), and hands both to add_job/3. Nothing about the queue lives in the caller — add("emails", …, connection: :redis) is a coordinate, not a container.

The add runs the matching script

add/4 builds a Job and runs the add* Lua script that matches its target. Each script atomically writes the job hash and pushes the job id onto the right state key, so an add is one atomic step — the hash and the membership land together or not at all. Pick a target to read its script.

add/4 → the matching add* script · pick a target

add_bulk/3 adds many jobs atomically in one MULTI/EXEC transaction. Whether one job or many, the write is atomic — the hash and the state-key membership are never half-applied.

What survives a restart

The interactive above is the whole point made concrete. A stateful object loses its in-memory job list when its process restarts; the jobs are gone. Redis loses nothing: the jobs persist in emq:{queue}:wait and the ZSETs, because Redis owns the durable state. A fresh add/4 call against the same name and connection re-attaches to the very same keys.

This is why the queue can be stateless. The durable thing is Redis, not the caller — so the caller can be a bare function call with no process behind it.

The keys are the queue — L1/L2

The Redis keys that are the queue — the wait LIST and the delayed / prioritized ZSETs — are L1, and the add* scripts that write them are L2. Both are frozen across runtimes. The component that calls them is L3/L4, and each runtime has its own.

the protocol (immutable L1/L2)

The keys that are the queue — emq:{q}:wait (LIST), emq:{q}:delayed / :prioritized (ZSETs) — and the add* scripts that write them. Frozen in every runtime.

its three runtimes (variable L3/L4)

Elixir's thin stateless Queue.add/4, Go's queue.Add, Node's queue.add — each a thin call onto the same keys and the same scripts. No runtime holds the queue; Redis does.

A queue name is a coordinate, not a container. The component is disposable; the keys are the queue.

Next: the worker fetch loop — the GenServer that pulls these jobs back off the keys, one atomic pickup at a time.

References

The Redis keys that hold the queue, and the BullMQ add API the stateless path mirrors.

Sources

BullMQ — Documentation — the queue API and the add scripts.
BullMQ — Home — the project EchoMQ implements.
Redis — Lists — the wait LIST a standard add pushes onto.
Redis — LPUSH — the push that admits a job to the queue.

Related in this course

E2.02 · Jobs, queues & workers — the module hub.
E2.02.1 · The job model — the job the queue admits.
E2.02.3 · The worker fetch loop — the worker that pulls from it.
E2.01.1 · The eight states — the state keys the add writes to.
redis-patterns R3 · Reliable queues — the queue-as-keys pattern, applied.