diff --git a/docs/architecture/ARCHITECTURE_OVERVIEW.md b/docs/architecture/ARCHITECTURE_OVERVIEW.md index 0b19f53..5cb51c6 100644 --- a/docs/architecture/ARCHITECTURE_OVERVIEW.md +++ b/docs/architecture/ARCHITECTURE_OVERVIEW.md @@ -21,7 +21,10 @@ Full homelab stack as of 2026-05-09. All 6 expansion phases are complete and dep - **Guest:** 10.5.90.0/24 - **Legacy quarantine:** 192.168.1.0/24 (`Old IoT`) - **UniFi policy zones:** `Internal` = Management + Trusted + Servers, `Untrusted` = IoT + Cameras + Old IoT, `Hotspot` = Guest -- **Current custom firewall hardening:** `Allow Internal to Untrusted` plus an explicit `Block Untrusted to Gateway Admin Surfaces` rule to keep IoT/Camera/Old IoT clients off the UDM Pro admin TCP ports while leaving the existing zone defaults in place +- **Current custom firewall hardening:** `Allow Internal to Untrusted`, `Block Untrusted to Gateway Admin Surfaces`, `Allow Untrusted to DNS Resolver`, `Block Untrusted to Gateway Default Services`, `Allow Untrusted to Gateway DHCP`, `Allow Untrusted to Gateway mDNS`, plus explicit public-DNS allows used for fallback +- **Current DHCP DNS policy:** `Management`, `Trusted`, `Servers`, `IoT`, `Camera`, and `Old IoT` advertise the Technitium trio `10.5.30.8`, `10.5.30.9`, and `10.5.30.10`, followed by external fallback `9.9.9.9`; `Guest` advertises `9.9.9.9` and `1.1.1.1` +- **Net effect:** every non-guest lane now has three independent internal Technitium resolvers for both public DNS and the private `home.paccoco.com` zone, plus external fallback if all homelab resolvers are down; untrusted lanes still retain DNS, DHCP, and mDNS but no longer have general gateway access or UDM Pro admin-surface access +- **Current DNS caveat:** the backup Technitium nodes on N.O.M.A.D. (`10.5.30.9`) and Serenity (`10.5.30.10`) start from a cloned PD Technitium config, so future authoritative-zone changes on PD must be resynced or automated to keep all three nodes aligned - **Tailscale:** Serenity reachable at 100.94.87.79 - **NFS:** Serenity exports `/mnt/user/data` and `/mnt/user/immich` → mounted on PD at `/mnt/unraid/` via post-init script - **Docker networking:** Cross-stack communication via named external networks (`ai-services`, `ix-databases_shared-databases`) diff --git a/docs/architecture/NETWORKING_MODEL.md b/docs/architecture/NETWORKING_MODEL.md index 4b22b02..de682f1 100644 --- a/docs/architecture/NETWORKING_MODEL.md +++ b/docs/architecture/NETWORKING_MODEL.md @@ -22,7 +22,17 @@ - `Hotspot` = Guest - **Current custom policy additions on top of UniFi defaults:** - `Allow Internal to Untrusted` - - `Block Untrusted to Gateway Admin Surfaces` (blocks IoT/Camera/Old IoT access to the UDM Pro admin TCP ports while leaving DHCP/DNS/mDNS/internet behavior to the existing zone defaults) + - `Block Untrusted to Gateway Admin Surfaces` + - `Allow Untrusted to DNS Resolver` + - `Block Untrusted to Gateway Default Services` + - `Allow Untrusted to Gateway DHCP` + - `Allow Untrusted to Gateway mDNS` + - explicit `Allow Public DNS` fallbacks +- **Current DHCP DNS policy:** + - `Management`, `Trusted`, `Servers`, `IoT`, `Camera`, and `Old IoT` advertise `10.5.30.8`, `10.5.30.9`, `10.5.30.10`, and `9.9.9.9` via DHCP + - `Guest` advertises `9.9.9.9` and `1.1.1.1` +- **Current hardening result:** every non-guest lane now has three internal Technitium resolvers for both public DNS and the private `home.paccoco.com` zone, plus an external fallback; IoT/Camera/Old IoT keep DNS, DHCP, and mDNS, but general `Untrusted -> Gateway` access and UDM Pro admin-surface access are blocked +- **Current DNS caveat:** the N.O.M.A.D. (`10.5.30.9`) and Serenity (`10.5.30.10`) Technitium backups are seeded from a cloned PD config. Any future authoritative-zone changes on PD must be resynced or automated so all three resolvers stay aligned - **Serenity IP:** 10.5.30.5 - **N.O.M.A.D. IP:** 10.5.30.7 - **PlausibleDeniability:** 10.5.30.6 (Servers VLAN) diff --git a/technitium-nomad/.env.example b/technitium-nomad/.env.example new file mode 100644 index 0000000..a689530 --- /dev/null +++ b/technitium-nomad/.env.example @@ -0,0 +1,18 @@ +TZ=America/Chicago +LAN_INTERFACE=enp5s0 +LAN_SUBNET=10.5.30.0/24 +LAN_GATEWAY=10.5.30.1 +TECHNITIUM_BIND_IP=10.5.30.9 +APPDATA_TECHNITIUM_ROOT=/opt/technitium-nomad/data +DNS_SERVER_DOMAIN=technitium-nomad.home.paccoco.com +DNS_SERVER_ADMIN_PASSWORD=CHANGE_ME +DNS_SERVER_SSO_ENABLED=false +DNS_SERVER_SSO_AUTHORITY= +DNS_SERVER_SSO_CLIENT_ID= +DNS_SERVER_SSO_CLIENT_SECRET= +DNS_SERVER_SSO_METADATA_ADDRESS= +DNS_SERVER_SSO_SCOPES=openid profile email groups +DNS_SERVER_SSO_ALLOW_SIGNUP=true +DNS_SERVER_SSO_ALLOW_SIGNUP_ONLY_FOR_MAPPED_USERS=true +DNS_SERVER_SSO_GROUP_MAP=authentik Admins=Administrators +DNS_SERVER_BLOCK_LIST_URLS=https://big.oisd.nl/ diff --git a/technitium-nomad/README.md b/technitium-nomad/README.md new file mode 100644 index 0000000..c541b6a --- /dev/null +++ b/technitium-nomad/README.md @@ -0,0 +1,39 @@ +# Technitium backup on N.O.M.A.D. + +Backup Technitium DNS node cloned from the PD pilot for Nomad. + +- Repo stack path: `/home/fizzlepoof/repos/truenas-stacks/technitium-nomad` +- Live stack path: `/opt/technitium-nomad` +- Bind IP: `10.5.30.9` +- DNS ports: `53/tcp`, `53/udp` +- Web console: `http://10.5.30.9/` +- Bootstrap domain if reset from empty config: `technitium-nomad.home.paccoco.com` + +This backup node is intended to replace Pi-hole fallback in DHCP resolver lists once validated. It reuses the PD Technitium config as the initial authoritative baseline. + +Current mirrored baseline from Pi-hole: +- blocklists: + - `https://raw.githubusercontent.com/StevenBlack/hosts/master/hosts` + - `https://big.oisd.nl` +- no custom local DNS entries detected +- no conditional forwarding detected +- no extra dnsmasq fragments detected + +Pilot-specific adaptation: +- Production Pi-hole forwards to PD-local Unbound on `127.0.0.1:5335`. +- Because the Technitium pilot uses a dedicated macvlan IP, that loopback-only upstream is not reachable from the pilot without changing production Unbound. +- For pilot safety, Technitium is seeded as a standalone recursive resolver instead of reconfiguring production Unbound. + +Notes: +- The stack binds only to `TECHNITIUM_BIND_IP`, so PD must have that secondary address present on `LAN_INTERFACE` before `docker compose up`. +- The helper script adds the pilot IP alias read/write on the live host for the current boot only. That is deliberate for pilot safety. +- Technitium reads the environment initialization only on first boot when `/etc/dns` is empty. If you need to re-seed initial settings, stop the container and clear the config directory first. +- SSO-capable deployments should set the `DNS_SERVER_SSO_*` variables. Keep a known local admin account as rollback because Technitium may only apply initial auth/env seeding cleanly on first boot or with an empty `/etc/dns` config dir. +- For native Authentik OIDC on this pilot, `DNS_SERVER_SSO_ALLOW_SIGNUP` must be `true` at least for the initial SSO user provisioning. Keeping `DNS_SERVER_SSO_ALLOW_SIGNUP_ONLY_FOR_MAPPED_USERS=true` preserves the gate so only users in mapped Authentik groups can auto-provision. +- Authentik native OIDC callback for Technitium is `https://dns.paccoco.com/sso/callback`. +- For Authentik native OIDC, the Technitium OAuth2 provider must have the default OpenID scope mappings attached: `openid`, `email`, and `profile`. Without them, Authentik will mint a token with no OpenID scopes and Technitium fails the `/application/o/userinfo/` call with `Scope mismatch` / `SSO authentication failed`. +- The authoritative zone `home.paccoco.com` is now hosted in Technitium with `dns.home.paccoco.com -> 10.5.30.8`. +- Clients will resolve `dns.home.paccoco.com` only when they query Technitium directly, or after DHCP/cutover points them at Technitium. + +- Host interface parent for macvlan: `enp5s0` +- Operational note: this node starts from a cloned Technitium config copied from PD; future record changes on PD must be resynced here until replication automation exists. diff --git a/technitium-nomad/bin/configure_technitium.py b/technitium-nomad/bin/configure_technitium.py new file mode 100644 index 0000000..f8e3000 --- /dev/null +++ b/technitium-nomad/bin/configure_technitium.py @@ -0,0 +1,160 @@ +#!/usr/bin/env python3 +import argparse +import json +import os +import sys +from typing import Any +from urllib import parse, request, error + + +def api_request(base_url: str, method: str, path: str, *, token: str | None = None, query: dict | None = None) -> tuple[int, Any]: + url = base_url.rstrip('/') + path + if query: + url += '?' + parse.urlencode(query, doseq=True) + headers = {'Accept': 'application/json'} + if token: + headers['Authorization'] = f'Bearer {token}' + req = request.Request(url, headers=headers, method=method) + try: + with request.urlopen(req, timeout=30) as resp: + return resp.status, json.load(resp) + except error.HTTPError as exc: + body = exc.read().decode('utf-8', 'replace') + try: + payload = json.loads(body) + except Exception: + payload = body + return exc.code, payload + + +def expect_ok(status, payload, context): + if status != 200 or payload.get('status') != 'ok': + print(f'{context} failed: http={status}', file=sys.stderr) + print(json.dumps(payload, indent=2, sort_keys=True), file=sys.stderr) + raise SystemExit(1) + + +def bool_s(v: bool) -> str: + return 'true' if v else 'false' + + +def main() -> int: + ap = argparse.ArgumentParser(description='Idempotent Technitium baseline config for the homelab pilot') + ap.add_argument('--base-url', default='http://127.0.0.1:5380') + ap.add_argument('--admin-user', default='admin') + ap.add_argument('--admin-pass', default=os.environ.get('DNS_SERVER_ADMIN_PASSWORD')) + ap.add_argument('--server-domain', default='dns.home.paccoco.com') + ap.add_argument('--zone', default='home.paccoco.com') + ap.add_argument('--record-host', default='dns.home.paccoco.com') + ap.add_argument('--record-ip', default='10.5.30.8') + ap.add_argument('--web-port', type=int, default=80) + ap.add_argument('--blocklist', action='append', default=None) + args = ap.parse_args() + + if not args.admin_pass: + print('Missing admin password: pass --admin-pass or set DNS_SERVER_ADMIN_PASSWORD', file=sys.stderr) + return 1 + + desired_blocklists = args.blocklist or ['https://big.oisd.nl/'] + + st, login = api_request(args.base_url, 'GET', '/api/user/login', query={ + 'user': args.admin_user, + 'pass': args.admin_pass, + 'includeInfo': 'true', + }) + expect_ok(st, login, 'login') + token = login['token'] + + st, settings = api_request(args.base_url, 'GET', '/api/settings/get', token=token) + expect_ok(st, settings, 'settings/get') + resp = settings['response'] + + query = { + 'dnsServerDomain': args.server_domain, + 'webServiceLocalAddresses': ','.join(resp.get('webServiceLocalAddresses') or ['[::]']), + 'webServiceHttpPort': str(args.web_port), + 'webServiceEnableTls': bool_s(False), + 'webServiceTlsPort': str(resp.get('webServiceTlsPort', 53443)), + 'webServiceHttpToTlsRedirect': bool_s(False), + 'webServiceUseSelfSignedTlsCertificate': bool_s(False), + 'preferIPv6': bool_s(bool(resp.get('preferIPv6', False))), + 'dnssecValidation': bool_s(True), + 'enableBlocking': bool_s(True), + 'allowTxtBlockingReport': bool_s(True), + 'blockListUrls': ','.join(desired_blocklists), + 'blockListUpdateIntervalHours': str(resp.get('blockListUpdateIntervalHours', 24)), + 'logQueries': bool_s(False), + 'allowRecursion': bool_s(True), + 'allowRecursionOnlyForPrivateNetworks': bool_s(True), + 'qnameMinimization': bool_s(True), + 'serveStale': bool_s(True), + } + + st, updated = api_request(args.base_url, 'GET', '/api/settings/set', token=token, query=query) + expect_ok(st, updated, 'settings/set') + + verify_base_url = args.base_url + parsed_base = parse.urlparse(args.base_url) + if parsed_base.port and parsed_base.port != args.web_port: + host = parsed_base.hostname or '127.0.0.1' + verify_base_url = f'{parsed_base.scheme}://{host}:{args.web_port}' + + st, zones = api_request(verify_base_url, 'GET', '/api/zones/list', token=token) + expect_ok(st, zones, 'zones/list') + zone_names = {z['name'] for z in zones['response']['zones']} + if args.zone not in zone_names: + st, created = api_request(verify_base_url, 'GET', '/api/zones/create', token=token, query={ + 'zone': args.zone, + 'type': 'Primary', + }) + expect_ok(st, created, 'zones/create') + + st, current_records = api_request(verify_base_url, 'GET', '/api/zones/records/get', token=token, query={ + 'domain': args.record_host, + 'zone': args.zone, + 'listZone': 'false', + }) + expect_ok(st, current_records, 'zones/records/get') + for record in current_records['response'].get('records', []): + if record.get('type') == 'A': + ip = (record.get('rData') or {}).get('ipAddress') + if ip and ip != args.record_ip: + st, deleted = api_request(verify_base_url, 'GET', '/api/zones/records/delete', token=token, query={ + 'domain': args.record_host, + 'zone': args.zone, + 'type': 'A', + 'ipAddress': ip, + }) + expect_ok(st, deleted, 'zones/records/delete') + + st, added = api_request(verify_base_url, 'GET', '/api/zones/records/add', token=token, query={ + 'domain': args.record_host, + 'zone': args.zone, + 'type': 'A', + 'ipAddress': args.record_ip, + 'overwrite': 'true', + 'ttl': '300', + }) + expect_ok(st, added, 'zones/records/add') + + st, verify = api_request(verify_base_url, 'GET', '/api/settings/get', token=token) + expect_ok(st, verify, 'settings/get verify') + v = verify['response'] + summary = { + 'dnsServerDomain': v.get('dnsServerDomain'), + 'webServiceHttpPort': v.get('webServiceHttpPort'), + 'webServiceEnableTls': v.get('webServiceEnableTls'), + 'recursion': v.get('recursion'), + 'dnssecValidation': v.get('dnssecValidation'), + 'enableBlocking': v.get('enableBlocking'), + 'allowTxtBlockingReport': v.get('allowTxtBlockingReport'), + 'blockListUrls': v.get('blockListUrls'), + 'zone': args.zone, + 'record': {args.record_host: args.record_ip}, + } + print(json.dumps(summary, indent=2, sort_keys=True)) + return 0 + + +if __name__ == '__main__': + raise SystemExit(main()) diff --git a/technitium-nomad/bin/prepare_pd.sh b/technitium-nomad/bin/prepare_pd.sh new file mode 100755 index 0000000..7d501f6 --- /dev/null +++ b/technitium-nomad/bin/prepare_pd.sh @@ -0,0 +1,31 @@ +#!/usr/bin/env bash +set -euo pipefail + +STACK_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" +cd "$STACK_DIR" + +if [[ ! -f .env ]]; then + echo "Missing .env in $STACK_DIR" + exit 1 +fi + +set -a +source .env +set +a + +if [[ -z "${LAN_INTERFACE:-}" || -z "${LAN_SUBNET:-}" || -z "${LAN_GATEWAY:-}" || -z "${TECHNITIUM_BIND_IP:-}" || -z "${APPDATA_TECHNITIUM_ROOT:-}" ]]; then + echo "LAN_INTERFACE, LAN_SUBNET, LAN_GATEWAY, TECHNITIUM_BIND_IP, and APPDATA_TECHNITIUM_ROOT must be set in .env" + exit 1 +fi + +sudo mkdir -p "${APPDATA_TECHNITIUM_ROOT}/config" "${APPDATA_TECHNITIUM_ROOT}/logs" + +echo "Validating compose config" +sudo docker compose --env-file .env config >/tmp/technitium-pilot-compose.rendered.yaml + +echo "Rendered compose saved to /tmp/technitium-pilot-compose.rendered.yaml" + +if [[ "${1:-}" == "--up" ]]; then + echo "Starting Technitium pilot" + sudo docker compose --env-file .env up -d +fi diff --git a/technitium-nomad/docker-compose.yaml b/technitium-nomad/docker-compose.yaml new file mode 100644 index 0000000..732dce1 --- /dev/null +++ b/technitium-nomad/docker-compose.yaml @@ -0,0 +1,48 @@ +services: + technitium: + image: docker.io/technitium/dns-server@sha256:99c250f0ee494f2f31e09a76d83f8213bc4c5d6f106b0895cd5f327518469c48 + container_name: technitium-dns-pilot + hostname: technitium-dns-pilot + restart: unless-stopped + networks: + lan_macvlan: + ipv4_address: ${TECHNITIUM_BIND_IP} + environment: + DNS_SERVER_DOMAIN: ${DNS_SERVER_DOMAIN} + DNS_SERVER_ADMIN_PASSWORD: ${DNS_SERVER_ADMIN_PASSWORD} + DNS_SERVER_SSO_ENABLED: ${DNS_SERVER_SSO_ENABLED:-false} + DNS_SERVER_SSO_AUTHORITY: ${DNS_SERVER_SSO_AUTHORITY:-} + DNS_SERVER_SSO_CLIENT_ID: ${DNS_SERVER_SSO_CLIENT_ID:-} + DNS_SERVER_SSO_CLIENT_SECRET: ${DNS_SERVER_SSO_CLIENT_SECRET:-} + DNS_SERVER_SSO_METADATA_ADDRESS: ${DNS_SERVER_SSO_METADATA_ADDRESS:-} + DNS_SERVER_SSO_SCOPES: ${DNS_SERVER_SSO_SCOPES:-openid profile email groups} + DNS_SERVER_SSO_ALLOW_SIGNUP: ${DNS_SERVER_SSO_ALLOW_SIGNUP:-false} + DNS_SERVER_SSO_ALLOW_SIGNUP_ONLY_FOR_MAPPED_USERS: ${DNS_SERVER_SSO_ALLOW_SIGNUP_ONLY_FOR_MAPPED_USERS:-true} + DNS_SERVER_SSO_GROUP_MAP: ${DNS_SERVER_SSO_GROUP_MAP:-authentik Admins=Administrators} + DNS_SERVER_WEB_SERVICE_HTTP_PORT: "80" + DNS_SERVER_WEB_SERVICE_HTTPS_PORT: "443" + DNS_SERVER_WEB_SERVICE_ENABLE_HTTPS: "true" + DNS_SERVER_WEB_SERVICE_USE_SELF_SIGNED_CERT: "true" + DNS_SERVER_WEB_SERVICE_HTTP_TO_TLS_REDIRECT: "true" + DNS_SERVER_OPTIONAL_PROTOCOL_DNS_OVER_HTTP: "false" + DNS_SERVER_RECURSION: "AllowOnlyForPrivateNetworks" + DNS_SERVER_ENABLE_BLOCKING: "true" + DNS_SERVER_ALLOW_TXT_BLOCKING_REPORT: "true" + DNS_SERVER_BLOCK_LIST_URLS: ${DNS_SERVER_BLOCK_LIST_URLS} + DNS_SERVER_LOG_USING_LOCAL_TIME: "true" + DNS_SERVER_LOG_FOLDER_PATH: /var/log/technitium/dns + DNS_SERVER_LOG_MAX_LOG_FILE_DAYS: "30" + DNS_SERVER_STATS_MAX_STAT_FILE_DAYS: "30" + volumes: + - ${APPDATA_TECHNITIUM_ROOT}/config:/etc/dns + - ${APPDATA_TECHNITIUM_ROOT}/logs:/var/log/technitium/dns + +networks: + lan_macvlan: + driver: macvlan + driver_opts: + parent: ${LAN_INTERFACE} + ipam: + config: + - subnet: ${LAN_SUBNET} + gateway: ${LAN_GATEWAY} diff --git a/technitium-pilot/.env.example b/technitium-pilot/.env.example index 92f3947..756b715 100644 --- a/technitium-pilot/.env.example +++ b/technitium-pilot/.env.example @@ -12,7 +12,7 @@ DNS_SERVER_SSO_CLIENT_ID= DNS_SERVER_SSO_CLIENT_SECRET= DNS_SERVER_SSO_METADATA_ADDRESS= DNS_SERVER_SSO_SCOPES=openid profile email groups -DNS_SERVER_SSO_ALLOW_SIGNUP=false +DNS_SERVER_SSO_ALLOW_SIGNUP=true DNS_SERVER_SSO_ALLOW_SIGNUP_ONLY_FOR_MAPPED_USERS=true DNS_SERVER_SSO_GROUP_MAP=authentik Admins=Administrators DNS_SERVER_BLOCK_LIST_URLS=https://big.oisd.nl/ diff --git a/technitium-pilot/README.md b/technitium-pilot/README.md index 69153bb..5ac7f62 100644 --- a/technitium-pilot/README.md +++ b/technitium-pilot/README.md @@ -29,6 +29,10 @@ Notes: - The helper script adds the pilot IP alias read/write on the live host for the current boot only. That is deliberate for pilot safety. - Technitium reads the environment initialization only on first boot when `/etc/dns` is empty. If you need to re-seed initial settings, stop the container and clear the config directory first. - SSO-capable deployments should set the `DNS_SERVER_SSO_*` variables. Keep a known local admin account as rollback because Technitium may only apply initial auth/env seeding cleanly on first boot or with an empty `/etc/dns` config dir. +- For native Authentik OIDC on this pilot, `DNS_SERVER_SSO_ALLOW_SIGNUP` must be `true` at least for the initial SSO user provisioning. Keeping `DNS_SERVER_SSO_ALLOW_SIGNUP_ONLY_FOR_MAPPED_USERS=true` preserves the gate so only users in mapped Authentik groups can auto-provision. - Authentik native OIDC callback for Technitium is `https://dns.paccoco.com/sso/callback`. +- For Authentik native OIDC, the Technitium OAuth2 provider must have the default OpenID scope mappings attached: `openid`, `email`, and `profile`. Without them, Authentik will mint a token with no OpenID scopes and Technitium fails the `/application/o/userinfo/` call with `Scope mismatch` / `SSO authentication failed`. - The authoritative zone `home.paccoco.com` is now hosted in Technitium with `dns.home.paccoco.com -> 10.5.30.8`. - Clients will resolve `dns.home.paccoco.com` only when they query Technitium directly, or after DHCP/cutover points them at Technitium. +- `bin/sync_backup_nodes.sh` is the PD-side replication runner that pushes the live Technitium config to the Nomad (`10.5.30.9`) and Serenity (`10.5.30.10`) backup nodes, then recycles those backup stacks and verifies DNS answers. +- Preferred automation model: root cron on PD every 15 minutes, logging to `/mnt/tank/docker/appdata/technitium-pilot/logs/sync-backup-nodes.log`. diff --git a/technitium-pilot/bin/sync_backup_nodes.sh b/technitium-pilot/bin/sync_backup_nodes.sh new file mode 100755 index 0000000..d682393 --- /dev/null +++ b/technitium-pilot/bin/sync_backup_nodes.sh @@ -0,0 +1,192 @@ +#!/usr/bin/env bash +set -euo pipefail + +SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +STACK_DIR="$(cd "$SCRIPT_DIR/.." && pwd)" + +PD_CONFIG_ROOT="${PD_CONFIG_ROOT:-/mnt/tank/docker/appdata/technitium-pilot/config}" +LOG_ROOT="${LOG_ROOT:-/mnt/tank/docker/appdata/technitium-pilot/logs}" +DIG_BIN="${DIG_BIN:-/usr/bin/dig}" +RSYNC_BIN="${RSYNC_BIN:-/usr/bin/rsync}" +SSH_BIN="${SSH_BIN:-/usr/bin/ssh}" +DATE_BIN="${DATE_BIN:-/usr/bin/date}" +MKDIR_BIN="${MKDIR_BIN:-/usr/bin/mkdir}" + +SSH_OPTS=(-o BatchMode=yes -o ConnectTimeout=10 -o StrictHostKeyChecking=accept-new) +RSYNC_EXCLUDES=(--exclude cache.bin --exclude 'stats/' --exclude 'blocklists/') + +NOMAD_HOST="${NOMAD_HOST:-fizzlepoof@10.5.30.7}" +NOMAD_KEY="${NOMAD_KEY:-/home/truenas_admin/.ssh/nomad_technitium_sync_ed25519}" +NOMAD_STACK_ROOT="${NOMAD_STACK_ROOT:-/opt/technitium-nomad}" +NOMAD_DNS_IP="${NOMAD_DNS_IP:-10.5.30.9}" + +SERENITY_HOST="${SERENITY_HOST:-root@10.5.30.5}" +SERENITY_KEY="${SERENITY_KEY:-/home/truenas_admin/.ssh/serenity_backup_ed25519}" +SERENITY_STACK_ROOT="${SERENITY_STACK_ROOT:-/mnt/user/appdata/technitium-serenity}" +SERENITY_DNS_IP="${SERENITY_DNS_IP:-10.5.30.10}" + +DRY_RUN=false +ONLY_TARGET="all" +SKIP_RESTART=false + +usage() { + cat <<'EOF' +Usage: sync_backup_nodes.sh [--dry-run] [--target nomad|serenity|all] [--skip-restart] + +Push the live PD Technitium config to the Nomad and Serenity backup Technitium nodes, +then recycle their compose stacks and verify both private and public DNS answers. +EOF +} + +while [[ $# -gt 0 ]]; do + case "$1" in + --dry-run) + DRY_RUN=true + ;; + --target) + ONLY_TARGET="${2:-}" + shift + ;; + --skip-restart) + SKIP_RESTART=true + ;; + -h|--help) + usage + exit 0 + ;; + *) + echo "Unknown argument: $1" >&2 + usage >&2 + exit 1 + ;; + esac + shift +done + +if [[ ! -d "$PD_CONFIG_ROOT" ]]; then + echo "PD config root missing: $PD_CONFIG_ROOT" >&2 + exit 1 +fi + +if [[ ! -x "$RSYNC_BIN" ]]; then + echo "rsync missing at $RSYNC_BIN" >&2 + exit 1 +fi + +if [[ ! -x "$SSH_BIN" ]]; then + echo "ssh missing at $SSH_BIN" >&2 + exit 1 +fi + +if [[ ! -x "$DIG_BIN" ]]; then + echo "dig missing at $DIG_BIN" >&2 + exit 1 +fi + +$MKDIR_BIN -p "$LOG_ROOT" +STAMP="$($DATE_BIN -u +%Y-%m-%dT%H:%M:%SZ)" + +log() { + printf '[%s] %s\n' "$($DATE_BIN -u +%Y-%m-%dT%H:%M:%SZ)" "$*" +} + +remote_prepare_nomad() { + "$SSH_BIN" -i "$NOMAD_KEY" "${SSH_OPTS[@]}" "$NOMAD_HOST" \ + "sudo -n mkdir -p '$NOMAD_STACK_ROOT/data/config' '$NOMAD_STACK_ROOT/data/logs'" +} + +remote_prepare_serenity() { + "$SSH_BIN" -i "$SERENITY_KEY" "${SSH_OPTS[@]}" "$SERENITY_HOST" \ + "mkdir -p '$SERENITY_STACK_ROOT/data/config' '$SERENITY_STACK_ROOT/data/logs'" +} + +remote_restart_nomad() { + "$SSH_BIN" -i "$NOMAD_KEY" "${SSH_OPTS[@]}" "$NOMAD_HOST" \ + "printf '%s\n' '$STAMP' | sudo -n tee '$NOMAD_STACK_ROOT/data/last_pd_sync_utc.txt' >/dev/null && \ + cd '$NOMAD_STACK_ROOT' && \ + sudo -n docker compose --env-file .env down && \ + sudo -n docker compose --env-file .env up -d" +} + +remote_restart_serenity() { + "$SSH_BIN" -i "$SERENITY_KEY" "${SSH_OPTS[@]}" "$SERENITY_HOST" \ + "printf '%s\n' '$STAMP' > '$SERENITY_STACK_ROOT/data/last_pd_sync_utc.txt' && \ + cd '$SERENITY_STACK_ROOT' && \ + docker compose --env-file .env down && \ + docker compose --env-file .env up -d" +} + +verify_dns() { + local label="$1" + local dns_ip="$2" + local private_tmp="/tmp/technitium-sync-${label}-private.txt" + local public_tmp="/tmp/technitium-sync-${label}-public.txt" + local attempt + log "Verifying $label at $dns_ip" + for attempt in {1..10}; do + if "$DIG_BIN" +time=2 +tries=1 @"$dns_ip" dns.home.paccoco.com A >"$private_tmp" 2>/dev/null && \ + "$DIG_BIN" +time=2 +tries=1 @"$dns_ip" cloudflare.com A >"$public_tmp" 2>/dev/null && \ + grep -q 'status: NOERROR' "$private_tmp" && \ + grep -q 'status: NOERROR' "$public_tmp" && \ + grep -q '10.5.30.8' "$private_tmp"; then + return 0 + fi + sleep 2 + done + echo "Verification failed for $label ($dns_ip) after retries" >&2 + cat "$private_tmp" 2>/dev/null >&2 || true + echo '---' >&2 + cat "$public_tmp" 2>/dev/null >&2 || true + return 1 +} + +sync_nomad() { + log "Preparing Nomad target" + remote_prepare_nomad + local rsync_cmd=("$RSYNC_BIN" -a --delete "${RSYNC_EXCLUDES[@]}" -e "$SSH_BIN -i $NOMAD_KEY ${SSH_OPTS[*]}" --rsync-path="sudo -n $RSYNC_BIN" "$PD_CONFIG_ROOT/" "$NOMAD_HOST:$NOMAD_STACK_ROOT/data/config/") + if $DRY_RUN; then + rsync_cmd=("$RSYNC_BIN" -an --delete "${RSYNC_EXCLUDES[@]}" -e "$SSH_BIN -i $NOMAD_KEY ${SSH_OPTS[*]}" --rsync-path="sudo -n $RSYNC_BIN" "$PD_CONFIG_ROOT/" "$NOMAD_HOST:$NOMAD_STACK_ROOT/data/config/") + fi + log "Syncing Nomad config" + "${rsync_cmd[@]}" + if ! $DRY_RUN && ! $SKIP_RESTART; then + log "Recycling Nomad Technitium stack" + remote_restart_nomad + verify_dns nomad "$NOMAD_DNS_IP" + fi +} + +sync_serenity() { + log "Preparing Serenity target" + remote_prepare_serenity + local rsync_cmd=("$RSYNC_BIN" -a --delete "${RSYNC_EXCLUDES[@]}" -e "$SSH_BIN -i $SERENITY_KEY ${SSH_OPTS[*]}" "$PD_CONFIG_ROOT/" "$SERENITY_HOST:$SERENITY_STACK_ROOT/data/config/") + if $DRY_RUN; then + rsync_cmd=("$RSYNC_BIN" -an --delete "${RSYNC_EXCLUDES[@]}" -e "$SSH_BIN -i $SERENITY_KEY ${SSH_OPTS[*]}" "$PD_CONFIG_ROOT/" "$SERENITY_HOST:$SERENITY_STACK_ROOT/data/config/") + fi + log "Syncing Serenity config" + "${rsync_cmd[@]}" + if ! $DRY_RUN && ! $SKIP_RESTART; then + log "Recycling Serenity Technitium stack" + remote_restart_serenity + verify_dns serenity "$SERENITY_DNS_IP" + fi +} + +case "$ONLY_TARGET" in + all) + sync_nomad + sync_serenity + ;; + nomad) + sync_nomad + ;; + serenity) + sync_serenity + ;; + *) + echo "Invalid --target value: $ONLY_TARGET" >&2 + exit 1 + ;; +esac + +log "Technitium backup sync finished successfully" diff --git a/technitium-serenity/.env.example b/technitium-serenity/.env.example new file mode 100644 index 0000000..584e1ac --- /dev/null +++ b/technitium-serenity/.env.example @@ -0,0 +1,18 @@ +TZ=America/Chicago +LAN_INTERFACE=br0 +LAN_SUBNET=10.5.30.0/24 +LAN_GATEWAY=10.5.30.1 +TECHNITIUM_BIND_IP=10.5.30.10 +APPDATA_TECHNITIUM_ROOT=/mnt/user/appdata/technitium-serenity/data +DNS_SERVER_DOMAIN=technitium-serenity.home.paccoco.com +DNS_SERVER_ADMIN_PASSWORD=CHANGE_ME +DNS_SERVER_SSO_ENABLED=false +DNS_SERVER_SSO_AUTHORITY= +DNS_SERVER_SSO_CLIENT_ID= +DNS_SERVER_SSO_CLIENT_SECRET= +DNS_SERVER_SSO_METADATA_ADDRESS= +DNS_SERVER_SSO_SCOPES=openid profile email groups +DNS_SERVER_SSO_ALLOW_SIGNUP=true +DNS_SERVER_SSO_ALLOW_SIGNUP_ONLY_FOR_MAPPED_USERS=true +DNS_SERVER_SSO_GROUP_MAP=authentik Admins=Administrators +DNS_SERVER_BLOCK_LIST_URLS=https://big.oisd.nl/ diff --git a/technitium-serenity/README.md b/technitium-serenity/README.md new file mode 100644 index 0000000..5a7a4cd --- /dev/null +++ b/technitium-serenity/README.md @@ -0,0 +1,39 @@ +# Technitium backup on Serenity + +Backup Technitium DNS node cloned from the PD pilot for Serenity. + +- Repo stack path: `/home/fizzlepoof/repos/truenas-stacks/technitium-serenity` +- Live stack path: `/mnt/user/appdata/technitium-serenity` +- Bind IP: `10.5.30.10` +- DNS ports: `53/tcp`, `53/udp` +- Web console: `http://10.5.30.10/` +- Bootstrap domain if reset from empty config: `technitium-serenity.home.paccoco.com` + +This backup node is intended to replace Pi-hole fallback in DHCP resolver lists once validated. It reuses the PD Technitium config as the initial authoritative baseline. + +Current mirrored baseline from Pi-hole: +- blocklists: + - `https://raw.githubusercontent.com/StevenBlack/hosts/master/hosts` + - `https://big.oisd.nl` +- no custom local DNS entries detected +- no conditional forwarding detected +- no extra dnsmasq fragments detected + +Pilot-specific adaptation: +- Production Pi-hole forwards to PD-local Unbound on `127.0.0.1:5335`. +- Because the Technitium pilot uses a dedicated macvlan IP, that loopback-only upstream is not reachable from the pilot without changing production Unbound. +- For pilot safety, Technitium is seeded as a standalone recursive resolver instead of reconfiguring production Unbound. + +Notes: +- The stack binds only to `TECHNITIUM_BIND_IP`, so PD must have that secondary address present on `LAN_INTERFACE` before `docker compose up`. +- The helper script adds the pilot IP alias read/write on the live host for the current boot only. That is deliberate for pilot safety. +- Technitium reads the environment initialization only on first boot when `/etc/dns` is empty. If you need to re-seed initial settings, stop the container and clear the config directory first. +- SSO-capable deployments should set the `DNS_SERVER_SSO_*` variables. Keep a known local admin account as rollback because Technitium may only apply initial auth/env seeding cleanly on first boot or with an empty `/etc/dns` config dir. +- For native Authentik OIDC on this pilot, `DNS_SERVER_SSO_ALLOW_SIGNUP` must be `true` at least for the initial SSO user provisioning. Keeping `DNS_SERVER_SSO_ALLOW_SIGNUP_ONLY_FOR_MAPPED_USERS=true` preserves the gate so only users in mapped Authentik groups can auto-provision. +- Authentik native OIDC callback for Technitium is `https://dns.paccoco.com/sso/callback`. +- For Authentik native OIDC, the Technitium OAuth2 provider must have the default OpenID scope mappings attached: `openid`, `email`, and `profile`. Without them, Authentik will mint a token with no OpenID scopes and Technitium fails the `/application/o/userinfo/` call with `Scope mismatch` / `SSO authentication failed`. +- The authoritative zone `home.paccoco.com` is now hosted in Technitium with `dns.home.paccoco.com -> 10.5.30.8`. +- Clients will resolve `dns.home.paccoco.com` only when they query Technitium directly, or after DHCP/cutover points them at Technitium. + +- Host interface parent for macvlan: `br0` +- Operational note: this node starts from a cloned Technitium config copied from PD; future record changes on PD must be resynced here until replication automation exists. diff --git a/technitium-serenity/bin/configure_technitium.py b/technitium-serenity/bin/configure_technitium.py new file mode 100644 index 0000000..f8e3000 --- /dev/null +++ b/technitium-serenity/bin/configure_technitium.py @@ -0,0 +1,160 @@ +#!/usr/bin/env python3 +import argparse +import json +import os +import sys +from typing import Any +from urllib import parse, request, error + + +def api_request(base_url: str, method: str, path: str, *, token: str | None = None, query: dict | None = None) -> tuple[int, Any]: + url = base_url.rstrip('/') + path + if query: + url += '?' + parse.urlencode(query, doseq=True) + headers = {'Accept': 'application/json'} + if token: + headers['Authorization'] = f'Bearer {token}' + req = request.Request(url, headers=headers, method=method) + try: + with request.urlopen(req, timeout=30) as resp: + return resp.status, json.load(resp) + except error.HTTPError as exc: + body = exc.read().decode('utf-8', 'replace') + try: + payload = json.loads(body) + except Exception: + payload = body + return exc.code, payload + + +def expect_ok(status, payload, context): + if status != 200 or payload.get('status') != 'ok': + print(f'{context} failed: http={status}', file=sys.stderr) + print(json.dumps(payload, indent=2, sort_keys=True), file=sys.stderr) + raise SystemExit(1) + + +def bool_s(v: bool) -> str: + return 'true' if v else 'false' + + +def main() -> int: + ap = argparse.ArgumentParser(description='Idempotent Technitium baseline config for the homelab pilot') + ap.add_argument('--base-url', default='http://127.0.0.1:5380') + ap.add_argument('--admin-user', default='admin') + ap.add_argument('--admin-pass', default=os.environ.get('DNS_SERVER_ADMIN_PASSWORD')) + ap.add_argument('--server-domain', default='dns.home.paccoco.com') + ap.add_argument('--zone', default='home.paccoco.com') + ap.add_argument('--record-host', default='dns.home.paccoco.com') + ap.add_argument('--record-ip', default='10.5.30.8') + ap.add_argument('--web-port', type=int, default=80) + ap.add_argument('--blocklist', action='append', default=None) + args = ap.parse_args() + + if not args.admin_pass: + print('Missing admin password: pass --admin-pass or set DNS_SERVER_ADMIN_PASSWORD', file=sys.stderr) + return 1 + + desired_blocklists = args.blocklist or ['https://big.oisd.nl/'] + + st, login = api_request(args.base_url, 'GET', '/api/user/login', query={ + 'user': args.admin_user, + 'pass': args.admin_pass, + 'includeInfo': 'true', + }) + expect_ok(st, login, 'login') + token = login['token'] + + st, settings = api_request(args.base_url, 'GET', '/api/settings/get', token=token) + expect_ok(st, settings, 'settings/get') + resp = settings['response'] + + query = { + 'dnsServerDomain': args.server_domain, + 'webServiceLocalAddresses': ','.join(resp.get('webServiceLocalAddresses') or ['[::]']), + 'webServiceHttpPort': str(args.web_port), + 'webServiceEnableTls': bool_s(False), + 'webServiceTlsPort': str(resp.get('webServiceTlsPort', 53443)), + 'webServiceHttpToTlsRedirect': bool_s(False), + 'webServiceUseSelfSignedTlsCertificate': bool_s(False), + 'preferIPv6': bool_s(bool(resp.get('preferIPv6', False))), + 'dnssecValidation': bool_s(True), + 'enableBlocking': bool_s(True), + 'allowTxtBlockingReport': bool_s(True), + 'blockListUrls': ','.join(desired_blocklists), + 'blockListUpdateIntervalHours': str(resp.get('blockListUpdateIntervalHours', 24)), + 'logQueries': bool_s(False), + 'allowRecursion': bool_s(True), + 'allowRecursionOnlyForPrivateNetworks': bool_s(True), + 'qnameMinimization': bool_s(True), + 'serveStale': bool_s(True), + } + + st, updated = api_request(args.base_url, 'GET', '/api/settings/set', token=token, query=query) + expect_ok(st, updated, 'settings/set') + + verify_base_url = args.base_url + parsed_base = parse.urlparse(args.base_url) + if parsed_base.port and parsed_base.port != args.web_port: + host = parsed_base.hostname or '127.0.0.1' + verify_base_url = f'{parsed_base.scheme}://{host}:{args.web_port}' + + st, zones = api_request(verify_base_url, 'GET', '/api/zones/list', token=token) + expect_ok(st, zones, 'zones/list') + zone_names = {z['name'] for z in zones['response']['zones']} + if args.zone not in zone_names: + st, created = api_request(verify_base_url, 'GET', '/api/zones/create', token=token, query={ + 'zone': args.zone, + 'type': 'Primary', + }) + expect_ok(st, created, 'zones/create') + + st, current_records = api_request(verify_base_url, 'GET', '/api/zones/records/get', token=token, query={ + 'domain': args.record_host, + 'zone': args.zone, + 'listZone': 'false', + }) + expect_ok(st, current_records, 'zones/records/get') + for record in current_records['response'].get('records', []): + if record.get('type') == 'A': + ip = (record.get('rData') or {}).get('ipAddress') + if ip and ip != args.record_ip: + st, deleted = api_request(verify_base_url, 'GET', '/api/zones/records/delete', token=token, query={ + 'domain': args.record_host, + 'zone': args.zone, + 'type': 'A', + 'ipAddress': ip, + }) + expect_ok(st, deleted, 'zones/records/delete') + + st, added = api_request(verify_base_url, 'GET', '/api/zones/records/add', token=token, query={ + 'domain': args.record_host, + 'zone': args.zone, + 'type': 'A', + 'ipAddress': args.record_ip, + 'overwrite': 'true', + 'ttl': '300', + }) + expect_ok(st, added, 'zones/records/add') + + st, verify = api_request(verify_base_url, 'GET', '/api/settings/get', token=token) + expect_ok(st, verify, 'settings/get verify') + v = verify['response'] + summary = { + 'dnsServerDomain': v.get('dnsServerDomain'), + 'webServiceHttpPort': v.get('webServiceHttpPort'), + 'webServiceEnableTls': v.get('webServiceEnableTls'), + 'recursion': v.get('recursion'), + 'dnssecValidation': v.get('dnssecValidation'), + 'enableBlocking': v.get('enableBlocking'), + 'allowTxtBlockingReport': v.get('allowTxtBlockingReport'), + 'blockListUrls': v.get('blockListUrls'), + 'zone': args.zone, + 'record': {args.record_host: args.record_ip}, + } + print(json.dumps(summary, indent=2, sort_keys=True)) + return 0 + + +if __name__ == '__main__': + raise SystemExit(main()) diff --git a/technitium-serenity/bin/prepare_pd.sh b/technitium-serenity/bin/prepare_pd.sh new file mode 100755 index 0000000..7d501f6 --- /dev/null +++ b/technitium-serenity/bin/prepare_pd.sh @@ -0,0 +1,31 @@ +#!/usr/bin/env bash +set -euo pipefail + +STACK_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" +cd "$STACK_DIR" + +if [[ ! -f .env ]]; then + echo "Missing .env in $STACK_DIR" + exit 1 +fi + +set -a +source .env +set +a + +if [[ -z "${LAN_INTERFACE:-}" || -z "${LAN_SUBNET:-}" || -z "${LAN_GATEWAY:-}" || -z "${TECHNITIUM_BIND_IP:-}" || -z "${APPDATA_TECHNITIUM_ROOT:-}" ]]; then + echo "LAN_INTERFACE, LAN_SUBNET, LAN_GATEWAY, TECHNITIUM_BIND_IP, and APPDATA_TECHNITIUM_ROOT must be set in .env" + exit 1 +fi + +sudo mkdir -p "${APPDATA_TECHNITIUM_ROOT}/config" "${APPDATA_TECHNITIUM_ROOT}/logs" + +echo "Validating compose config" +sudo docker compose --env-file .env config >/tmp/technitium-pilot-compose.rendered.yaml + +echo "Rendered compose saved to /tmp/technitium-pilot-compose.rendered.yaml" + +if [[ "${1:-}" == "--up" ]]; then + echo "Starting Technitium pilot" + sudo docker compose --env-file .env up -d +fi diff --git a/technitium-serenity/docker-compose.yaml b/technitium-serenity/docker-compose.yaml new file mode 100644 index 0000000..742079a --- /dev/null +++ b/technitium-serenity/docker-compose.yaml @@ -0,0 +1,43 @@ +services: + technitium: + image: docker.io/technitium/dns-server@sha256:99c250f0ee494f2f31e09a76d83f8213bc4c5d6f106b0895cd5f327518469c48 + container_name: technitium-dns-pilot + hostname: technitium-dns-pilot + restart: unless-stopped + networks: + lan_macvlan: + ipv4_address: ${TECHNITIUM_BIND_IP} + environment: + DNS_SERVER_DOMAIN: ${DNS_SERVER_DOMAIN} + DNS_SERVER_ADMIN_PASSWORD: ${DNS_SERVER_ADMIN_PASSWORD} + DNS_SERVER_SSO_ENABLED: ${DNS_SERVER_SSO_ENABLED:-false} + DNS_SERVER_SSO_AUTHORITY: ${DNS_SERVER_SSO_AUTHORITY:-} + DNS_SERVER_SSO_CLIENT_ID: ${DNS_SERVER_SSO_CLIENT_ID:-} + DNS_SERVER_SSO_CLIENT_SECRET: ${DNS_SERVER_SSO_CLIENT_SECRET:-} + DNS_SERVER_SSO_METADATA_ADDRESS: ${DNS_SERVER_SSO_METADATA_ADDRESS:-} + DNS_SERVER_SSO_SCOPES: ${DNS_SERVER_SSO_SCOPES:-openid profile email groups} + DNS_SERVER_SSO_ALLOW_SIGNUP: ${DNS_SERVER_SSO_ALLOW_SIGNUP:-false} + DNS_SERVER_SSO_ALLOW_SIGNUP_ONLY_FOR_MAPPED_USERS: ${DNS_SERVER_SSO_ALLOW_SIGNUP_ONLY_FOR_MAPPED_USERS:-true} + DNS_SERVER_SSO_GROUP_MAP: ${DNS_SERVER_SSO_GROUP_MAP:-authentik Admins=Administrators} + DNS_SERVER_WEB_SERVICE_HTTP_PORT: "80" + DNS_SERVER_WEB_SERVICE_HTTPS_PORT: "443" + DNS_SERVER_WEB_SERVICE_ENABLE_HTTPS: "true" + DNS_SERVER_WEB_SERVICE_USE_SELF_SIGNED_CERT: "true" + DNS_SERVER_WEB_SERVICE_HTTP_TO_TLS_REDIRECT: "true" + DNS_SERVER_OPTIONAL_PROTOCOL_DNS_OVER_HTTP: "false" + DNS_SERVER_RECURSION: "AllowOnlyForPrivateNetworks" + DNS_SERVER_ENABLE_BLOCKING: "true" + DNS_SERVER_ALLOW_TXT_BLOCKING_REPORT: "true" + DNS_SERVER_BLOCK_LIST_URLS: ${DNS_SERVER_BLOCK_LIST_URLS} + DNS_SERVER_LOG_USING_LOCAL_TIME: "true" + DNS_SERVER_LOG_FOLDER_PATH: /var/log/technitium/dns + DNS_SERVER_LOG_MAX_LOG_FILE_DAYS: "30" + DNS_SERVER_STATS_MAX_STAT_FILE_DAYS: "30" + volumes: + - ${APPDATA_TECHNITIUM_ROOT}/config:/etc/dns + - ${APPDATA_TECHNITIUM_ROOT}/logs:/var/log/technitium/dns + +networks: + lan_macvlan: + external: true + name: br0