# Known Quirks

Per-service gotchas that aren't bugs but will bite you if you forget them.

## PlausibleDeniability

### n8n workflow 16 (NFS Watchdog)
- Runs `sudo docker restart` and `sudo /usr/bin/bash .../mount-unraid-nfs.sh` over SSH from n8n container
- Requires a NOPASSWD sudoers rule or SSH commands will hang waiting for a password:
  ```
  truenas_admin ALL=(ALL) NOPASSWD: /usr/bin/docker, /usr/bin/bash /mnt/tank/docker/scripts/mount-unraid-nfs.sh
  ```
  Add via `sudo visudo -f /etc/sudoers.d/n8n-watchdog`
- **Must use `/usr/bin/bash` not `/bin/bash`** — on TrueNAS SCALE, bash is at `/usr/bin/bash`. The sudoers rule and the workflow command must match exactly or sudo will prompt for a password.
- SSH credential in n8n must use key-based auth (see n8n-workflows/README.md for keygen steps)

### immich-ml
- Healthcheck has no curl/wget — uses `/proc/net/tcp6` pattern
- Cache goes to `/mnt/docker-ssd/docker/appdata/immich-ml` (separate from immich-server's appdata)

### rackpeek
- No curl/wget in image — healthcheck uses `/proc/net/tcp6`

### donetick
- Env vars alone are insufficient for DB type selection
- Requires `/mnt/tank/docker/appdata/donetick/config/selfhosted.yaml` with full DB config
- Uses viper config loader — `DT_ENV` controls config file path
- Public exposure through Pangolin can stay unhealthy after renumbering if the target health-check hostname is stale even when the target IP/port are already fixed; verify both `ip` and `hcHostname`
- If PD-side Pangolin/Newt targets still probe a stale pre-renumbering host IP (for example `10.5.1.6`) but the backend is otherwise healthy, the fastest reversible recovery is a runtime `/32` compatibility alias on PD while the authoritative Pangolin target/health-check state is corrected
- If the PD `newt-loopback-bridge` helper is using `network_mode: "container:<newt>"`, restarting `ix-newt-newt-1` can leave the loopback relays broken until `newt-loopback-bridge` is also restarted; symptom is public 502/503 on `localhost`-backed Pangolin routes even after target health turns green
- OIDC metadata for the frontend is exposed from `/api/v1/resource`; if the login button is missing, check that endpoint before debugging the SPA

### homepage
- Live config is `/mnt/tank/docker/appdata/homepage/services.yaml`; it is not currently repo-managed, so live edits should be mirrored back into docs when they matter operationally
- `books.paccoco.com` is the working public Calibre-Web route; `calibre.paccoco.com` / `kindle.paccoco.com` are legacy/broken unless separate Pangolin resources are created for them
- RoMm widget URLs must use a backend Homepage can actually reach from PD; if the only working path is the auth-gated public Pangolin route, remove the widget instead of leaving a stale literal LAN IP

### shlink
- Data directory must be `chmod 777` — runs as non-root user that doesn't match host default ownership

### openwebui
- DB: user=`openwebui`, db=`openwebui` on shared-postgres
- If restart-looping with auth errors: container wasn't on `ix-databases_shared-databases` at creation time — must `down && up`

### plex
- Port `5353/udp` conflicts with system avahi/mDNS — remove from port mappings

### scriberr (shelved)
- SQLite incompatible with ZFS nfsv4 ACLs (SQLITE_CANTOPEN error 14)
- Revisit when Postgres support available in CUDA image

### meshmonitor (auto-upgrade)
- `AUTO_UPGRADE_ENABLED=true` causes crashes on TrueNAS if bind mount source directories don't exist
- The upgrader container mounts compose dir as `/compose` — relative paths resolve differently
- Fix: pre-create all bind mount source directories before deploying

### Any container using `down && up` requirement
- Network attachments happen at container creation, not restart
- `docker restart` or `docker compose restart` will NOT fix missing network attachments
- Must use `docker compose --env-file .env down <service> && docker compose --env-file .env up -d <service>`

---

## Serenity

### qBit Mover Script
- Pauses torrents 0–4 days old before running mover to prevent partial file moves
- MAM-init script updates MyAnonamouse IP inside qbit container on startup

### Unraid-Cloudflared-Tunnel
- Dead container, should be removed

### Serenity Newt / Pangolin stale health-check IP drift
- Several Serenity Newt-backed Pangolin targets were corrected to `ip=localhost`/`10.5.30.5` but still retained stale `hcHostname=10.5.1.5`, so Newt health checks failed with `no route to host`
- Verified affected target IDs on 2026-05-25: `15`, `20`, `25`, `29`, `56`, `57`, `58`, `69`
- A temporary local `/32` alias on Serenity (`ip addr add 10.5.1.5/32 dev br0`) proved the diagnosis by flipping multiple targets back to healthy
- That alias was then removed because the old `10.5.1.5` address is no longer allowed on the VLAN
- After removal, targets immediately fell back to unhealthy on the same stale health-check URLs, confirming the real problem is authoritative Pangolin metadata drift, not local app failure
- Treat the alias only as a diagnostic probe; do not persist it. The correct fix is to rewrite Pangolin health-check hostnames away from the stale pre-renumbering IP.

---

## N.O.M.A.D.

### NVIDIA Container Toolkit
- Repo has invalid GPG key — remove `/etc/apt/sources.list.d/*nvidia*` if apt errors

### Pelican / Wings
- Interactive artisan commands break over SSH (stty issue) — always use inline flags:
  `--email=x --username=y --password=z --admin=1`
- Wings CORS derived from `remote:` URL, not `allowed_origins` — keep remote as `https://`
- `panel.paccoco.com` in `/etc/hosts` → `127.0.0.1` for NAT loopback
- Wings uses `--ignore-certificate-errors` for local self-signed cert

### Port 8080
- Occupied by `nomad_admin` container — Wings uses 8443 instead