truenas-stacks/docs/troubleshooting/KNOWN_QUIRKS.md

Known Quirks

Per-service gotchas that aren't bugs but will bite you if you forget them.

PlausibleDeniability

n8n workflow 16 (NFS Watchdog)

• Runs sudo docker restart and sudo /usr/bin/bash .../mount-unraid-nfs.sh over SSH from n8n container
• Requires a NOPASSWD sudoers rule or SSH commands will hang waiting for a password:

  truenas_admin ALL=(ALL) NOPASSWD: /usr/bin/docker, /usr/bin/bash /mnt/tank/docker/scripts/mount-unraid-nfs.sh
  

  Add via sudo visudo -f /etc/sudoers.d/n8n-watchdog
• Must use /usr/bin/bash not /bin/bash — on TrueNAS SCALE, bash is at /usr/bin/bash. The sudoers rule and the workflow command must match exactly or sudo will prompt for a password.
• SSH credential in n8n must use key-based auth (see n8n-workflows/README.md for keygen steps)

immich-ml

• Healthcheck has no curl/wget — uses /proc/net/tcp6 pattern
• Cache goes to /mnt/docker-ssd/docker/appdata/immich-ml (separate from immich-server's appdata)

rackpeek

• No curl/wget in image — healthcheck uses /proc/net/tcp6

donetick

• Env vars alone are insufficient for DB type selection
• Requires /mnt/tank/docker/appdata/donetick/config/selfhosted.yaml with full DB config
• Uses viper config loader — DT_ENV controls config file path
• Public exposure through Pangolin can stay unhealthy after renumbering if the target health-check hostname is stale even when the target IP/port are already fixed; verify both ip and hcHostname
• If PD-side Pangolin/Newt targets still probe a stale pre-renumbering host IP (for example 10.5.1.6) but the backend is otherwise healthy, the fastest reversible recovery is a runtime /32 compatibility alias on PD while the authoritative Pangolin target/health-check state is corrected
• If the PD newt-loopback-bridge helper is using network_mode: "container:<newt>", restarting ix-newt-newt-1 can leave the loopback relays broken until newt-loopback-bridge is also restarted; symptom is public 502/503 on localhost-backed Pangolin routes even after target health turns green
• OIDC metadata for the frontend is exposed from /api/v1/resource; if the login button is missing, check that endpoint before debugging the SPA

homepage

• Live config is /mnt/tank/docker/appdata/homepage/services.yaml; it is not currently repo-managed, so live edits should be mirrored back into docs when they matter operationally
• books.paccoco.com is the working public Calibre-Web route; calibre.paccoco.com / kindle.paccoco.com are legacy/broken unless separate Pangolin resources are created for them
• RoMm widget URLs must use a backend Homepage can actually reach from PD; if the only working path is the auth-gated public Pangolin route, remove the widget instead of leaving a stale literal LAN IP

shlink

• Data directory must be chmod 777 — runs as non-root user that doesn't match host default ownership

openwebui

• DB: user=openwebui, db=openwebui on shared-postgres
• If restart-looping with auth errors: container wasn't on ix-databases_shared-databases at creation time — must down && up

plex

• Port 5353/udp conflicts with system avahi/mDNS — remove from port mappings

scriberr (shelved)

• SQLite incompatible with ZFS nfsv4 ACLs (SQLITE_CANTOPEN error 14)
• Revisit when Postgres support available in CUDA image

meshmonitor (auto-upgrade)

• AUTO_UPGRADE_ENABLED=true causes crashes on TrueNAS if bind mount source directories don't exist
• The upgrader container mounts compose dir as /compose — relative paths resolve differently
• Fix: pre-create all bind mount source directories before deploying

Any container using down && up requirement

• Network attachments happen at container creation, not restart
• docker restart or docker compose restart will NOT fix missing network attachments
• Must use docker compose --env-file .env down <service> && docker compose --env-file .env up -d <service>

---

Serenity

qBit Mover Script

• Pauses torrents 0–4 days old before running mover to prevent partial file moves
• MAM-init script updates MyAnonamouse IP inside qbit container on startup

Unraid-Cloudflared-Tunnel

• Older notes called this dead, but live inspection on 2026-05-25 showed active Cloudflare QUIC edge registrations and a configured tunnel token.
• Follow-up audit showed it is only transport-alive: current-run metrics reported zero proxied requests, its remote-managed ingress config still points at legacy 192.168.1.x origins, and the listed public hostnames appear to be served elsewhere now.
• Treat it as removable stale Cloudflare deadwood after a brief stop/remove observation window; preserve appdata during the cooling-off period and clean up Cloudflare-side tunnel config separately.

Serenity Wave 1 cleanup guardrails

• Do not assume Unraid-Cloudflared-Tunnel is dead just because it looked obsolete in older notes; live inspection on 2026-05-25 showed active Cloudflare tunnel registrations, so audit usage before removal.
• Do not remove Serenity's Pi-hole HA containers blindly while 10.5.30.53 still answers DNS; verify the full Technitium cutover path first.
• Recent Created containers on Unraid may represent intentionally retained templates rather than stale debris; distinguish them from long-dead exited containers before pruning.

Serenity Newt / Pangolin stale health-check IP drift

• Several Serenity Newt-backed Pangolin targets were corrected to ip=localhost/10.5.30.5 but still retained stale hcHostname=10.5.1.5, so Newt health checks failed with no route to host
• Verified affected target IDs on 2026-05-25: 15, 20, 25, 29, 56, 57, 58, 69
• A temporary local /32 alias on Serenity (ip addr add 10.5.1.5/32 dev br0) proved the diagnosis by flipping multiple targets back to healthy
• That alias was then removed because the old 10.5.1.5 address is no longer allowed on the VLAN
• After removal, targets immediately fell back to unhealthy on the same stale health-check URLs, confirming the real problem is authoritative Pangolin metadata drift, not local app failure
• Treat the alias only as a diagnostic probe; do not persist it. The correct fix is to rewrite Pangolin health-check hostnames away from the stale pre-renumbering IP.
• Authoritative fix completed for the Serenity audit set on 2026-05-25: targets 15, 20, 25, 29, 56, 57, 58, and 69 now use 10.5.30.5 for both routing and health checks, and no audited Serenity Pangolin target still carries 10.5.1.5 in live API state.

---

N.O.M.A.D.

NVIDIA Container Toolkit

• Repo has invalid GPG key — remove /etc/apt/sources.list.d/*nvidia* if apt errors

Pelican / Wings

• Interactive artisan commands break over SSH (stty issue) — always use inline flags: --email=x --username=y --password=z --admin=1
• Wings CORS derived from remote: URL, not allowed_origins — keep remote as https://
• panel.paccoco.com in /etc/hosts → 127.0.0.1 for NAT loopback
• Wings uses --ignore-certificate-errors for local self-signed cert

Port 8080

• Occupied by nomad_admin container — Wings uses 8443 instead