Refresh Serenity placement plan after live review

docs/planning/PD_FUTURE_STATE_ARCHITECTURE.md

@@ -200,6 +200,13 @@ Optional elsewhere:
 - heavy inference on Rocinante if PD does not get the 4090
 - high-risk security experiments on a future dedicated lab node if the school/lab workload outgrows PD-hosted segmented VMs
 
+### Move off Serenity early; does not need the storage cutover
+- Wizarr -> PD or retire if no longer useful
+- Hawser + dockersocket -> PD or retire as a pair
+- netdata -> keep only where it still adds monitoring value
+
+These are low-risk peel-off candidates because they are not part of the storage-local torrent/media locality lane.
+
 ## Cybersecurity VM model on PD
 
 ### This is acceptable on PD
@@ -251,7 +258,7 @@ Suggested policy posture:
 ### Phase 1: document and reduce Serenity sprawl now
 - document current host roles and future-state design
 - classify Serenity services into keep / move / retire
-- move low-risk non-storage-tied apps off Serenity first
+- move low-risk non-storage-tied apps off Serenity first (`Wizarr`, `Hawser` + `dockersocket`, and possibly `netdata`)
 
 ### Phase 2: build upgraded PD
 - install TrueNAS Scale on the new PD hardware
@@ -263,6 +270,7 @@ Suggested policy posture:
 - migrate any remaining general-purpose apps off Serenity
 - relocate databases/appdata onto the intended SSD tier
 - validate ingress, monitoring, and DNS roles
+- keep `Newt`, `reranker`, and the off-PD Technitium node as explicit special cases until their redesign work is ready
 
 ### Phase 4: move the storage-dependent torrent/media automation stack
 - after the storage move design is finalized, move qbit + ARR locality to PD
@@ -293,5 +301,6 @@ Suggested policy posture:
 
 For the current Serenity Docker review, the guiding rule is:
 - remove obvious sprawl from Serenity now
+- peel off the miscellaneous low-risk apps before touching the storage-locality lane
 - do not over-optimize the final placement around Serenity permanence
 - remember that once PD directly owns the storage, the qbit/ARR locality argument flips to PD

docs/planning/SERENITY_DOCKER_AUDIT.md

@@ -1,6 +1,6 @@
 # Serenity Docker Audit
 
-Status: live-audited baseline for cleanup and migration planning, refreshed after the GameVault/RomM cutover cleanup.
+Status: live-audited baseline for cleanup and migration planning, refreshed after the GameVault/RomM cutover cleanup and the broader keep/move/retire placement review.
 
 Last live verification: 2026-05-26
 
@@ -37,6 +37,7 @@ Common pattern observed from live mounts:
 - these stacks are bound heavily to `/mnt/user/data`
 - torrent state and backup artifacts live under Serenity-owned paths like `/mnt/user/data/torrents`, `/mnt/user/data/BT_backup`, and appdata under `/mnt/user/appdata/*`
 - current path locality still argues for keeping them on Serenity until PD directly owns the disks and final media paths
+- `shelfmark` is not a generic "random app" here; its live mounts also tie it to Serenity-owned `/mnt/user/data`, audiobook ingest paths, and torrent/library paths
 
 ### Keep temporarily, but plan to move or collapse later
 These are live today, but they are not good long-term reasons to keep Serenity alive after PD is rebuilt.
@@ -44,6 +45,7 @@ These are live today, but they are not good long-term reasons to keep Serenity a
 - `reranker`
   - currently on Serenity because CPU-only TEI was moved off PD
   - planned long-term home: PD if PD remains the AI control-plane/core host
+  - live runtime uses `/mnt/user/appdate/reranker`, not `/mnt/user/appdata/reranker`; treat that as a migration hazard that must be handled deliberately
 - `technitium-dns-pilot`
   - current backup Technitium node on Serenity (`10.5.30.10`)
   - long-term: keep at least one off-PD backup resolver, but that does not have to remain on Serenity forever
@@ -52,11 +54,61 @@ These are live today, but they are not good long-term reasons to keep Serenity a
   - operator confirmed it is still needed, so it should be preserved during cleanup and only rehomed deliberately later
 - `Hawser`
   - helper/observability tooling, not a reason to preserve Serenity as a host role
+- `dockersocket`
+  - Hawser helper sidecar only; keep or move with Hawser rather than treating it as a standalone host role
 - `netdata`
   - useful while Serenity remains live, not a reason to keep Serenity permanently
 - `Wizarr`
   - this is a normal app workload, not a storage-appliance-only workload
-  - long-term candidates for PD once the storage and app-host migration is ready
+  - long-term candidate for PD well before any final Serenity storage retirement if you want to peel off low-risk miscellaneous apps early
+
+### Placement verdict by service group
+
+#### Keep on Serenity until the storage-locality cutover is designed
+- torrent / download lane:
+  - `qbit`
+  - `GluetunVPN`
+  - `qbit_manage`
+  - `autobrr`
+  - `unpackerr`
+- media automation lane:
+  - `prowlarr`
+  - `sonarr`
+  - `sonarr-anime`
+  - `radarr`
+  - `lidarr`
+  - `readarr`
+  - `readarr-epub`
+  - `bazarr`
+- locality-adjacent helpers:
+  - `Notifiarr`
+  - `shelfmark`
+
+Reason:
+- all of these either mount Serenity-owned `/mnt/user/data` directly or depend on the downloader/media-path locality that still lives on Serenity today
+- moving them before PD owns the storage locally would just trade one cleanup project for a fragile NFS-path rewrite project
+
+#### Move off Serenity whenever the destination design is ready
+- `Wizarr`
+- `Hawser`
+- `dockersocket`
+- `netdata`
+
+Reason:
+- these are not durable storage-owner roles
+- they can move independently of the qbit/ARR locality lane once you decide whether PD should absorb them or whether one of them is no longer worth keeping
+
+#### Keep as explicit temporary exceptions, then redesign last
+- `reranker`
+- `technitium-dns-pilot`
+- `Newt`
+
+Reason:
+- these are the remaining special cases that still justify Serenity for non-media reasons
+- each one has a real redesign question attached:
+  - reranker: whether PD should re-absorb AI helper services later
+  - Technitium backup node: which off-PD host should own the durable backup resolver role
+  - Newt: when Serenity-hosted Pangolin resources are deliberately rehomed so the local tunnel is no longer required
 
 ### Retire candidates
 These should be treated as cleanup targets unless a specific live dependency is rediscovered.
@@ -116,33 +168,25 @@ Examples from the live container inspection:
 - `shelfmark` binds `/mnt/user/data`, `/mnt/user/data/media/books/Audiobooks`, `/mnt/user/data/media/books/ingest`, and `/mnt/user/data/torrents`
 - most ARR-family services bind `/mnt/user/data`
 - `reranker` binds `/mnt/user/appdate/reranker -> /data`
+- `Wizarr` only binds its own appdata/database paths and has no meaningful storage-locality reason to stay on Serenity
+- `Hawser` + `dockersocket` are local helper tooling, not media-path owners
 
 Notable typo/risk:
 - `reranker` is mounted from `/mnt/user/appdate/reranker` (note `appdate`, not `appdata`)
-- verify whether that path is intentional or an unnoticed typo before migration work touches it
+- `/mnt/user/appdate/reranker` exists live; `/mnt/user/appdata/reranker` does not currently exist
+- treat that as an intentional-on-disk reality for now, but fix the naming deliberately during a future migration rather than by surprise during unrelated cleanup
 
-## Recommended cleanup order
+## Recommended next work order
 
-### Phase 1: remove obvious deadwood
-1. verify no dependency remains on `Unraid-Cloudflared-Tunnel`
-2. inspect whether the legacy Pi-hole stack still serves any traffic or admin purpose
-3. remove stale created/exited containers that have no active role:
-   - `calibre-web`
-   - `calibre`
-   - `SuggestArr`
-   - `Cleanuparr`
-   - `agregarr`
-   - `Huntarr`
-   - `omegabrr`
+### Phase 1: completed cleanup baseline
+- dead Cloudflared / Pi-hole / stale-container cleanup is done
+- the old local DB pair and source GameVault/RomM containers are already retired
+- current live work should start from the narrower remaining set, not re-open that cleanup unless new evidence appears
 
-### Phase 2: identify silent database dependencies
-Before touching `postgresql15` or `MariaDB-Official`, map which containers point at them.
-
-Questions to answer:
-- which apps on Serenity still use local Postgres?
-- which apps on Serenity still use local MariaDB?
-- are GameVault, RomM, Shelfmark, or Wizarr still backed by these local DBs?
-- can any of those apps be moved to PD's shared DB stack cleanly?
+### Phase 2: peel off the low-risk non-locality apps
+- move `Wizarr` to PD when convenient
+- move or retire `Hawser` + `dockersocket`
+- decide whether `netdata` still provides unique value once the broader monitoring stack is considered
 
 ### Phase 3: preserve the only things that currently justify Serenity
 Until PD owns the storage locally, keep the torrent/media-ingest locality group together:
@@ -156,33 +200,36 @@ Until PD owns the storage locally, keep the torrent/media-ingest locality group
 ### Phase 4: cut over after PD storage migration
 Once PD directly owns the relevant media/torrent datasets:
 - move qbit + ARR locality to PD
-- move reranker to PD if desired
+- keep path locality on the box that owns the disks
+- move or retire the locality-adjacent helpers that were intentionally left with that lane
+
+### Phase 5: redesign the special cases last
+- move reranker to PD if desired, while normalizing the `appdate`/`appdata` path naming intentionally
 - keep at least one off-PD Technitium node somewhere, preferably NOMAD if Serenity is retiring
-- move or retire the miscellaneous app workloads still left on Serenity
-- shut Serenity down as a permanent app host
+- re-home or retire Serenity-local `Newt` only after Pangolin dependencies are deliberately redesigned
 
 ## Kanban-ready workstreams
 
 ### Epic A — Serenity live inventory normalization
-- capture `docker ps`, `docker ps -a`, mounts, ports, and database dependencies
-- map every live container to one of: keep-now, move-to-PD, retire-now, or remove-as-stale
-- verify whether any runtime definitions exist only in Unraid templates and not in repo-managed compose
+- capture `docker ps`, mounts, ports, and remaining host-role assumptions
+- keep the repo docs aligned with the actual Unraid runtime inventory
+- treat ad-hoc Unraid container definitions as a drift risk until they are intentionally replaced
 
-### Epic B — obvious dead container cleanup
-- remove dead Cloudflared tunnel
-- remove stale created/exited clutter containers
-- verify and retire legacy Pi-hole/Keepalived/Unbound stack if no hidden dependency remains
+### Epic B — low-risk miscellaneous app peel-off
+- move `Wizarr` to PD or retire it
+- decide whether `Hawser` + `dockersocket` should move as a pair or just disappear
+- decide whether `netdata` still earns its keep alongside the broader monitoring stack
 
-### Epic C — database dependency audit
-- identify which Serenity apps use `postgresql15`
-- identify which Serenity apps use `MariaDB-Official`
-- decide whether those DBs move to PD shared databases or are retired with the apps
-
-### Epic D — torrent/media-locality preservation until PD cutover
+### Epic C — torrent/media-locality preservation until PD cutover
 - keep qbit/ARR stack stable on Serenity for now
 - document exact media/torrent paths that must exist on PD before migration
 - prevent premature moves that would recreate NFS-path weirdness
 
+### Epic D — special-case redesign
+- normalize the reranker path oddity during a planned move rather than an incidental cleanup
+- decide where the off-PD backup Technitium role should live after Serenity
+- re-home Pangolin/Newt dependencies last, not during the media cutover
+
 ### Epic E — final Serenity retirement
 - move remaining wanted apps to PD or NOMAD
 - preserve only the intended backup-DNS failure-domain role off PD
@@ -200,13 +247,14 @@ Resolved on 2026-05-25:
    - move qbit + ARR family to PD after storage cutover
    - leave no intentional production app role on Serenity
    - retire Serenity entirely
+6. `Wizarr`, `Hawser`/`dockersocket`, and probably `netdata` are explicitly low-risk move/retire candidates that do not need to wait for the storage cutover.
 
 ## Remaining verification questions
 
-- Confirm whether anything beyond `GameVault` still depends on `postgresql15`.
-- Confirm whether anything beyond `RomM` still depends on `MariaDB-Official`.
-- Is `/mnt/user/appdate/reranker` intentional, or a typo that should be corrected before migration?
-- When Pangolin API write auth is available again, rewrite Serenity target health-check hostnames away from stale `10.5.1.5` so the runtime alias can be removed cleanly.
+- No additional live dependency beyond the retired `GameVault`/`RomM` pair was surfaced during the quick DB audit; if that changes later, treat it as a rediscovery against rollback artifacts rather than a live-stack blocker.
+- Decide when the retained DB/appdata rollback artifacts are old enough to archive more aggressively or finally delete.
+- Normalize the `reranker` path naming during the eventual move: preserve the live `/mnt/user/appdate/reranker` data now, but decide whether the destination should standardize back to `appdata`.
+- Serenity's Pangolin/Newt health-check drift is fixed now; any future Newt rehome is an architecture task, not a stale-health-check incident response.
 
 ## Pangolin / Newt live remediation status
 
@@ -264,7 +312,7 @@ Rollback / evidence artifacts:
 
 ## Database dependency findings
 
-Live inspection of container environments currently points to:
+Live inspection before the GameVault/RomM retirement pointed to:
 
 - `GameVault` -> local `postgresql15`
   - `DB_HOST=10.5.30.5`