Files
truenas-stacks/docs/troubleshooting/KNOWN_QUIRKS.md
Fizzlepoof eba18beabe
Some checks failed
secret-guardrails / artifact-secret-scan (push) Has been cancelled
secret-guardrails / gitleaks (push) Has been cancelled
docs: record Serenity wave1 guardrails
2026-05-25 21:50:55 +00:00

104 lines
6.7 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# Known Quirks
Per-service gotchas that aren't bugs but will bite you if you forget them.
## PlausibleDeniability
### n8n workflow 16 (NFS Watchdog)
- Runs `sudo docker restart` and `sudo /usr/bin/bash .../mount-unraid-nfs.sh` over SSH from n8n container
- Requires a NOPASSWD sudoers rule or SSH commands will hang waiting for a password:
```
truenas_admin ALL=(ALL) NOPASSWD: /usr/bin/docker, /usr/bin/bash /mnt/tank/docker/scripts/mount-unraid-nfs.sh
```
Add via `sudo visudo -f /etc/sudoers.d/n8n-watchdog`
- **Must use `/usr/bin/bash` not `/bin/bash`** — on TrueNAS SCALE, bash is at `/usr/bin/bash`. The sudoers rule and the workflow command must match exactly or sudo will prompt for a password.
- SSH credential in n8n must use key-based auth (see n8n-workflows/README.md for keygen steps)
### immich-ml
- Healthcheck has no curl/wget — uses `/proc/net/tcp6` pattern
- Cache goes to `/mnt/docker-ssd/docker/appdata/immich-ml` (separate from immich-server's appdata)
### rackpeek
- No curl/wget in image — healthcheck uses `/proc/net/tcp6`
### donetick
- Env vars alone are insufficient for DB type selection
- Requires `/mnt/tank/docker/appdata/donetick/config/selfhosted.yaml` with full DB config
- Uses viper config loader — `DT_ENV` controls config file path
- Public exposure through Pangolin can stay unhealthy after renumbering if the target health-check hostname is stale even when the target IP/port are already fixed; verify both `ip` and `hcHostname`
- If PD-side Pangolin/Newt targets still probe a stale pre-renumbering host IP (for example `10.5.1.6`) but the backend is otherwise healthy, the fastest reversible recovery is a runtime `/32` compatibility alias on PD while the authoritative Pangolin target/health-check state is corrected
- If the PD `newt-loopback-bridge` helper is using `network_mode: "container:<newt>"`, restarting `ix-newt-newt-1` can leave the loopback relays broken until `newt-loopback-bridge` is also restarted; symptom is public 502/503 on `localhost`-backed Pangolin routes even after target health turns green
- OIDC metadata for the frontend is exposed from `/api/v1/resource`; if the login button is missing, check that endpoint before debugging the SPA
### homepage
- Live config is `/mnt/tank/docker/appdata/homepage/services.yaml`; it is not currently repo-managed, so live edits should be mirrored back into docs when they matter operationally
- `books.paccoco.com` is the working public Calibre-Web route; `calibre.paccoco.com` / `kindle.paccoco.com` are legacy/broken unless separate Pangolin resources are created for them
- RoMm widget URLs must use a backend Homepage can actually reach from PD; if the only working path is the auth-gated public Pangolin route, remove the widget instead of leaving a stale literal LAN IP
### shlink
- Data directory must be `chmod 777` — runs as non-root user that doesn't match host default ownership
### openwebui
- DB: user=`openwebui`, db=`openwebui` on shared-postgres
- If restart-looping with auth errors: container wasn't on `ix-databases_shared-databases` at creation time — must `down && up`
### plex
- Port `5353/udp` conflicts with system avahi/mDNS — remove from port mappings
### scriberr (shelved)
- SQLite incompatible with ZFS nfsv4 ACLs (SQLITE_CANTOPEN error 14)
- Revisit when Postgres support available in CUDA image
### meshmonitor (auto-upgrade)
- `AUTO_UPGRADE_ENABLED=true` causes crashes on TrueNAS if bind mount source directories don't exist
- The upgrader container mounts compose dir as `/compose` — relative paths resolve differently
- Fix: pre-create all bind mount source directories before deploying
### Any container using `down && up` requirement
- Network attachments happen at container creation, not restart
- `docker restart` or `docker compose restart` will NOT fix missing network attachments
- Must use `docker compose --env-file .env down <service> && docker compose --env-file .env up -d <service>`
---
## Serenity
### qBit Mover Script
- Pauses torrents 04 days old before running mover to prevent partial file moves
- MAM-init script updates MyAnonamouse IP inside qbit container on startup
### Unraid-Cloudflared-Tunnel
- Older notes called this dead, but live inspection on 2026-05-25 showed active Cloudflare QUIC edge registrations and a configured tunnel token.
- Treat it as a live service pending usage audit, not a safe blind-removal candidate.
### Serenity Wave 1 cleanup guardrails
- Do not assume `Unraid-Cloudflared-Tunnel` is dead just because it looked obsolete in older notes; live inspection on 2026-05-25 showed active Cloudflare tunnel registrations, so audit usage before removal.
- Do not remove Serenity's Pi-hole HA containers blindly while `10.5.30.53` still answers DNS; verify the full Technitium cutover path first.
- Recent `Created` containers on Unraid may represent intentionally retained templates rather than stale debris; distinguish them from long-dead exited containers before pruning.
### Serenity Newt / Pangolin stale health-check IP drift
- Several Serenity Newt-backed Pangolin targets were corrected to `ip=localhost`/`10.5.30.5` but still retained stale `hcHostname=10.5.1.5`, so Newt health checks failed with `no route to host`
- Verified affected target IDs on 2026-05-25: `15`, `20`, `25`, `29`, `56`, `57`, `58`, `69`
- A temporary local `/32` alias on Serenity (`ip addr add 10.5.1.5/32 dev br0`) proved the diagnosis by flipping multiple targets back to healthy
- That alias was then removed because the old `10.5.1.5` address is no longer allowed on the VLAN
- After removal, targets immediately fell back to unhealthy on the same stale health-check URLs, confirming the real problem is authoritative Pangolin metadata drift, not local app failure
- Treat the alias only as a diagnostic probe; do not persist it. The correct fix is to rewrite Pangolin health-check hostnames away from the stale pre-renumbering IP.
- Authoritative fix completed for the Serenity audit set on 2026-05-25: targets `15`, `20`, `25`, `29`, `56`, `57`, `58`, and `69` now use `10.5.30.5` for both routing and health checks, and no audited Serenity Pangolin target still carries `10.5.1.5` in live API state.
---
## N.O.M.A.D.
### NVIDIA Container Toolkit
- Repo has invalid GPG key — remove `/etc/apt/sources.list.d/*nvidia*` if apt errors
### Pelican / Wings
- Interactive artisan commands break over SSH (stty issue) — always use inline flags:
`--email=x --username=y --password=z --admin=1`
- Wings CORS derived from `remote:` URL, not `allowed_origins` — keep remote as `https://`
- `panel.paccoco.com` in `/etc/hosts` → `127.0.0.1` for NAT loopback
- Wings uses `--ignore-certificate-errors` for local self-signed cert
### Port 8080
- Occupied by `nomad_admin` container — Wings uses 8443 instead