129 lines
4.5 KiB
Markdown
129 lines
4.5 KiB
Markdown
# DNS Resilience
|
|
|
|
Current homelab DNS resilience model after the single-node outage lesson.
|
|
|
|
## Problem Statement
|
|
|
|
When Technitium on PD died, clients that depended on that single internal resolver lost DNS. Public internet connectivity did not matter because name resolution was gone first.
|
|
|
|
The fix was to stop treating DNS as a single-host service and advertise multiple internal resolvers across the important LANs.
|
|
|
|
## Current Resolver Topology
|
|
|
|
| Resolver | Host | Service role | Resolver IP |
|
|
|----------|------|--------------|-------------|
|
|
| PD Technitium | PlausibleDeniability | primary live source of truth | `10.5.30.8` |
|
|
| Nomad Technitium | N.O.M.A.D. | backup internal resolver | `10.5.30.9` |
|
|
| Serenity Technitium | Serenity | backup internal resolver | `10.5.30.10` |
|
|
| Quad9 | external | public-DNS fallback only | `9.9.9.9` |
|
|
|
|
All three Technitium nodes serve:
|
|
- recursive public DNS
|
|
- the private authoritative zone `home.paccoco.com`
|
|
|
|
## DHCP Advertisement Policy
|
|
|
|
UniFi DHCP currently advertises the Technitium trio plus an external fallback on:
|
|
- `Management`
|
|
- `Trusted`
|
|
- `Servers`
|
|
- `IoT`
|
|
- `Camera`
|
|
- `Old IoT`
|
|
|
|
Advertised order:
|
|
1. `10.5.30.8`
|
|
2. `10.5.30.9`
|
|
3. `10.5.30.10`
|
|
4. `9.9.9.9`
|
|
|
|
`Guest` remains public-only:
|
|
- `9.9.9.9`
|
|
- `1.1.1.1`
|
|
|
|
## What This Fixes
|
|
|
|
- A single Technitium host failure no longer wipes DNS for Trusted or Servers clients.
|
|
- Internal-zone continuity now survives the loss of any one Technitium node, as long as at least one of the remaining Technitium nodes is healthy.
|
|
- Public DNS continuity still exists even if the whole internal Technitium trio is down, because clients also have `9.9.9.9`.
|
|
|
|
## Important Limits
|
|
|
|
### External fallback is not full internal-DNS continuity
|
|
|
|
`9.9.9.9` is only a public fallback.
|
|
|
|
It can preserve general internet resolution if all internal resolvers die, but it does not replace the authoritative internal `home.paccoco.com` zone behavior.
|
|
|
|
Operationally: if all three Technitium nodes are down, clients may still resolve public domains, but private homelab names may be missing or semantically wrong.
|
|
|
|
### Macvlan health checks are deceptive
|
|
|
|
These Technitium nodes use dedicated macvlan IPs.
|
|
|
|
That means a host often cannot reliably query its own resolver IP:
|
|
- PD may fail to query `10.5.30.8` from the PD host itself
|
|
- NOMAD may fail to query `10.5.30.9` from NOMAD itself
|
|
- Serenity may fail to query `10.5.30.10` from Serenity itself
|
|
|
|
Do not treat same-host DNS failures as proof that the resolver is down.
|
|
|
|
Use an off-host probe from another LAN host or client.
|
|
|
|
## Current Sync Model
|
|
|
|
PD is the authoritative Technitium source of truth.
|
|
|
|
Live sync runner:
|
|
- path: `/mnt/docker-ssd/docker/compose/technitium-pilot/bin/sync_backup_nodes.sh`
|
|
|
|
Live schedule:
|
|
- root cron on PD every 15 minutes
|
|
|
|
Log path:
|
|
- `/mnt/tank/docker/appdata/technitium-pilot/logs/sync-backup-nodes.log`
|
|
|
|
The runner does all of the following:
|
|
1. rsync the live PD Technitium config to NOMAD and Serenity
|
|
2. exclude rebuildable cache/stats/blocklist artifacts
|
|
3. restart the backup stacks
|
|
4. verify a private authoritative answer for `dns.home.paccoco.com`
|
|
5. verify a public recursive answer for `cloudflare.com`
|
|
6. stamp each backup with `last_pd_sync_utc.txt`
|
|
|
|
## Validation SOP
|
|
|
|
When verifying DNS resilience, check from the correct network context.
|
|
|
|
Minimum proof:
|
|
1. confirm UniFi DHCP advertises `10.5.30.8`, `10.5.30.9`, `10.5.30.10`, and `9.9.9.9` on the intended VLANs
|
|
2. from an off-host client or peer host, query each internal resolver directly for:
|
|
- `dns.home.paccoco.com A`
|
|
- `cloudflare.com A`
|
|
3. confirm the PD sync cron exists
|
|
4. inspect the latest sync log for successful Nomad and Serenity verification
|
|
|
|
Recommended direct checks:
|
|
- `dig @10.5.30.8 dns.home.paccoco.com A`
|
|
- `dig @10.5.30.9 dns.home.paccoco.com A`
|
|
- `dig @10.5.30.10 dns.home.paccoco.com A`
|
|
- `dig @10.5.30.8 cloudflare.com A`
|
|
- `dig @10.5.30.9 cloudflare.com A`
|
|
- `dig @10.5.30.10 cloudflare.com A`
|
|
|
|
## Operator Guidance
|
|
|
|
- If internal-zone records change on PD, the backups should pick them up automatically within 15 minutes.
|
|
- If the sync log shows failures, treat that as an internal-zone resilience regression even if public DNS still works.
|
|
- If testing from the host that owns the macvlan IP fails, repeat the test from another host before escalating.
|
|
- Keep at least one external fallback in DHCP for non-guest lanes.
|
|
|
|
## Source Paths
|
|
|
|
- Repo stack docs:
|
|
- `technitium-pilot/README.md`
|
|
- `technitium-nomad/README.md`
|
|
- `technitium-serenity/README.md`
|
|
- Architecture docs:
|
|
- `docs/architecture/ARCHITECTURE_OVERVIEW.md`
|
|
- `docs/architecture/NETWORKING_MODEL.md` |