jonnifyelixir
F7 · Deploy to Fly / F7.2 · runtime config / F7.2.3 · name resolution

F7.2 · deep dive 3 · .internal vs .flycast & the discriminator

F7.2.3

The two private names & the discriminator

Fly exposes two private-name schemes for reaching one app from another, and choosing the wrong one — or the right one for a stopped instance — produces the same Postgrex :nxdomain as the address-family fault from dive 2. .internal is direct 6PN: org-scoped, AAAA-only, and it returns a record only for an app with a live machine in the requester’s 6PN. .flycast reaches an app through Fly’s internal proxy and additionally needs a flycast IP. Two schemes, two failure modes, one error string. This dive gives the test that tells them apart: getent hosts <db>.internal run from another running machine in the org.

We read the first bring-up’s record as built: the prod.exs fly_dns topology polls echo-portal.internal, the DB was reached at portal-postgres.internal, and the cross-machine getent test settled which of two identical-looking crashes was the real one.

Two private names, one org

An app on Fly reaches another app over the private IPv6 network (6PN) by a private name, and there are two name schemes. .internal is direct 6PN. A name like portal-postgres.internal is org-scoped, carries only AAAA (IPv6) records, and resolves only for an app that has a live machine in the requester’s 6PN. It requires the two apps to sit in the same organisation. This is the convention the deploy uses: the prod.exs fly_dns topology polls echo-portal.internal to form the cluster, and the first bring-up reached the database at portal-postgres.internal. When that database is auto-stopped, its .internal name returns no record, and the boot crashes with :nxdomain — this is gotcha 2.

.flycast is the Fly-proxy private name. It reaches an app through Fly’s internal proxy rather than directly, and it additionally needs a flycast IP — for a Fly Managed Postgres, that IP is allocated by fly mpg attach. The first bring-up’s portal-postgres is a same-org postgres-flex HA cluster reached over .internal, not a Managed Postgres over .flycast; F7.3 names that discriminator so the right host scheme is chosen for the right kind of database. Pick the database kind to read which scheme its host carries.

Host-scheme chooser · which private name reaches this database?

SAME-ORG POSTGRES-FLEX (HA) PRIVATE-NAME SCHEME .internal — direct 6PN ADDRESS FAMILY AAAA-only (IPv6) SAME ORG REQUIRED yes EXTRA STEP none

Both private names carry only AAAA records, so both demand an inet6 socket; the scheme they differ on is direct 6PN versus the Fly proxy — and the database kind, not a guess, picks which.

One symptom, two faults, the discriminator

The trap is that two different faults raise the same error. Gotcha 2 — the name has no record because the instance is stopped or out of org — crashes Postgrex with :nxdomain. Gotcha 3 — the name has an AAAA record but the app A-queries it on the default :inet family (dive 2’s fault) — crashes with the identical :nxdomain. The error string does not say which. The test that isolates them is a single command run from another running machine in the org: getent hosts <db>.internal. That command uses glibc getaddrinfo, the same resolution path Erlang’s native lookup takes. If the other machine resolves the name to an AAAA, the name and DNS are sound — so the fault is the app’s address family (gotcha 3, fixed in runtime.exs), not the network (gotcha 2).

The first bring-up ran exactly this test: getent hosts portal-postgres.internal SUCCEEDED from inside the running, same-org portal-postgres machine, returning an AAAA. The name was sound, so the app’s IPv4 default was the fault — which is how gotcha 3 was identified as the decisive one. A complementary check guards the other direction: if even echo-portal.internal — the app’s OWN name — fails to resolve from inside a machine, the machine’s 6PN DNS itself is broken, not the target. Set the database host’s machine state and the app’s Postgrex address family below and read the diagnosis the discriminator returns for that cell.

:nxdomain discriminator · which fault is it, and what does getent say?

THE APP’S BOOT Postgrex connect — ? getent hosts <db>.internal (other machine) — THE DISCRIMINATOR POINTS AT gotcha 3 — wrong address family fixed in runtime.exs: socket_options: [:inet6]

The error string is the same in every failing cell; the discriminator is what differs — a successful getent from another machine clears the network and convicts the app’s address family.

The IPv6 literal, named as a diagnostic

One more unblock is worth naming for completeness, and then setting aside. A bracketed IPv6 literal in DATABASE_URL — a fixed address rather than a name — sidesteps resolution entirely: with no DNS lookup, neither gotcha 2 nor gotcha 3 can fire. It confirms the network path works. But it pins the URL to a machine IP that Fly can recreate, so it is brittle as a deploy value. The durable fix is the one dive 2 wired: socket_options: [:inet6] on Portal.Repo, so the .internal AAAA resolves over IPv6 through a name that survives a machine being recreated. The literal is a diagnostic, not the deploy path.

# DATABASE_URL — a bracketed IPv6 literal sidesteps DNS (diagnostic only)
# fixed address, no resolution — pins the URL to a recreatable machine IP
DATABASE_URL="ecto://portal:****@[fdaa:0:1:a7b:1c2:3:4:2]:5432/portal"

# the durable path — resolve the AAAA-only .internal name over IPv6
# runtime.exs consumes ECTO_IPV6 → socket_options on Portal.Repo
DATABASE_URL="ecto://portal:****@portal-postgres.internal:5432/portal"  # + socket_options: [:inet6]

a name that has no record

gotcha 2 — the instance is stopped or out of org, so .internal returns nothing and the boot :nxdomains.

a name with a record the app cannot read

gotcha 3 — the AAAA exists, but a default :inet A-query returns the same :nxdomain; the discriminator separates the two.

The config now reads its secrets, binds the proxy-reachable port, speaks IPv6 to an AAAA-only host, and carries the test that tells two identical crashes apart. What it has not yet done is reach a database with a schema in it: F7.3, where the database goes live, brings up the migration and gotcha 4 over at F7 · Deploy to Fly.

References

The two private-name schemes, the same-org HA Postgres reached over the direct one, and the glibc resolver path the cross-machine discriminator and Erlang’s native lookup share.

Sources

Related in this course