docs: capture Serenity majority migration plan

docs/planning/PD_FUTURE_STATE_ARCHITECTURE.md

@@ -164,7 +164,6 @@ Media / library:
 - Seerr
 - RomM
 - GameVault
-- Wizarr
 - Shelfmark
 - Notifiarr
 
@@ -201,8 +200,8 @@ Optional elsewhere:
 - high-risk security experiments on a future dedicated lab node if the school/lab workload outgrows PD-hosted segmented VMs
 
 ### Move off Serenity early; does not need the storage cutover
-- Wizarr -> PD or retire if no longer useful
-- Hawser + dockersocket -> PD or retire as a pair
+- retire Wizarr if it is still present; rebuild later only if onboarding need returns
+- Notifiarr -> PD only if PD headroom makes the move worth it
 - netdata -> keep only where it still adds monitoring value
 
 These are low-risk peel-off candidates because they are not part of the storage-local torrent/media locality lane.
@@ -258,7 +257,7 @@ Suggested policy posture:
 ### Phase 1: document and reduce Serenity sprawl now
 - document current host roles and future-state design
 - classify Serenity services into keep / move / retire
-- move low-risk non-storage-tied apps off Serenity first (`Wizarr`, `Hawser` + `dockersocket`, and possibly `netdata`)
+- retire or peel off only the smallest low-risk items first (`Wizarr`, maybe `Notifiarr`, and possibly `netdata`) while leaving `Hawser` + `dockersocket` in place as the Dockhand management path
 
 ### Phase 2: build upgraded PD
 - install TrueNAS Scale on the new PD hardware
@@ -279,8 +278,8 @@ Suggested policy posture:
 
 ### Phase 5: move or redesign the remaining special cases
 - move reranker onto PD if AI core lands there
-- redesign backup DNS so Serenity is no longer required
-- preserve at least one off-PD Technitium node on NOMAD
+- redesign backup DNS so Serenity is no longer required, likely by introducing a small dedicated Raspberry Pi 4B resolver lane while preserving an off-PD Technitium failure domain
+- preserve at least one off-PD Technitium node outside PD even after Serenity retires
 
 ### Phase 6: retire Serenity
 - verify pool adoption is complete
@@ -301,6 +300,6 @@ Suggested policy posture:
 
 For the current Serenity Docker review, the guiding rule is:
 - remove obvious sprawl from Serenity now
-- peel off the miscellaneous low-risk apps before touching the storage-locality lane
+- retire dead weight and peel off only the smallest miscellaneous low-risk apps before touching the storage-locality lane
 - do not over-optimize the final placement around Serenity permanence
 - remember that once PD directly owns the storage, the qbit/ARR locality argument flips to PD

docs/planning/SERENITY_DOCKER_AUDIT.md

@@ -30,7 +30,6 @@ These are the containers whose current placement still makes operational sense b
 - `bazarr`
 - `autobrr`
 - `unpackerr`
-- `Notifiarr`
 - `shelfmark`
 
 Common pattern observed from live mounts:
@@ -53,14 +52,17 @@ These are live today, but they are not good long-term reasons to keep Serenity a
   - live on Serenity now
   - operator confirmed it is still needed, so it should be preserved during cleanup and only rehomed deliberately later
 - `Hawser`
-  - helper/observability tooling, not a reason to preserve Serenity as a host role
+  - operator confirmed Dockhand depends on it for remote management of Serenity's remaining Docker stacks
+  - keep on Serenity with `dockersocket` until that management pattern is intentionally replaced
 - `dockersocket`
-  - Hawser helper sidecar only; keep or move with Hawser rather than treating it as a standalone host role
+  - Hawser helper sidecar required by the current Dockhand remote-management path
 - `netdata`
   - useful while Serenity remains live, not a reason to keep Serenity permanently
+- `Notifiarr`
+  - low-risk utility app that can move later if PD has spare headroom
+  - not urgent while PD remains RAM-constrained
 - `Wizarr`
-  - this is a normal app workload, not a storage-appliance-only workload
-  - long-term candidate for PD well before any final Serenity storage retirement if you want to peel off low-risk miscellaneous apps early
+  - operator confirmed it is unused and should be retired rather than migrated
 
 ### Placement verdict by service group
 
@@ -88,15 +90,14 @@ Reason:
 - all of these either mount Serenity-owned `/mnt/user/data` directly or depend on the downloader/media-path locality that still lives on Serenity today
 - moving them before PD owns the storage locally would just trade one cleanup project for a fragile NFS-path rewrite project
 
-#### Move off Serenity whenever the destination design is ready
-- `Wizarr`
-- `Hawser`
-- `dockersocket`
-- `netdata`
+#### Optional early cleanup / peel-off work
+- retire `Wizarr`
+- move `Notifiarr` only if PD headroom clearly allows it
+- review whether `netdata` still adds enough local-monitoring value to keep
 
 Reason:
-- these are not durable storage-owner roles
-- they can move independently of the qbit/ARR locality lane once you decide whether PD should absorb them or whether one of them is no longer worth keeping
+- these are the few remaining low-risk app-level changes that do not depend on the qbit/ARR storage cutover
+- PD is constrained right now, so only very small wins should happen before the larger storage redesign
 
 #### Keep as explicit temporary exceptions, then redesign last
 - `reranker`
@@ -184,8 +185,8 @@ Notable typo/risk:
 - current live work should start from the narrower remaining set, not re-open that cleanup unless new evidence appears
 
 ### Phase 2: peel off the low-risk non-locality apps
-- move `Wizarr` to PD when convenient
-- move or retire `Hawser` + `dockersocket`
+- retire `Wizarr`
+- optionally move `Notifiarr` if PD headroom clearly permits it
 - decide whether `netdata` still provides unique value once the broader monitoring stack is considered
 
 ### Phase 3: preserve the only things that currently justify Serenity
@@ -216,8 +217,8 @@ Once PD directly owns the relevant media/torrent datasets:
 - treat ad-hoc Unraid container definitions as a drift risk until they are intentionally replaced
 
 ### Epic B — low-risk miscellaneous app peel-off
-- move `Wizarr` to PD or retire it
-- decide whether `Hawser` + `dockersocket` should move as a pair or just disappear
+- retire `Wizarr`
+- decide whether `Notifiarr` is worth an early move given PD RAM pressure
 - decide whether `netdata` still earns its keep alongside the broader monitoring stack
 
 ### Epic C — torrent/media-locality preservation until PD cutover
@@ -247,7 +248,7 @@ Resolved on 2026-05-25:
    - move qbit + ARR family to PD after storage cutover
    - leave no intentional production app role on Serenity
    - retire Serenity entirely
-6. `Wizarr`, `Hawser`/`dockersocket`, and probably `netdata` are explicitly low-risk move/retire candidates that do not need to wait for the storage cutover.
+6. `Wizarr` is explicit retire-now dead weight; `Notifiarr` is the main optional tiny move candidate; `Hawser`/`dockersocket` remain intentional keepers because Dockhand depends on them.
 
 ## Remaining verification questions
 

docs/planning/SERENITY_MAJORITY_MIGRATION_PLAN.md

@@ -0,0 +1,348 @@
+# Serenity Majority Migration Plan
+
+Status: approved planning baseline for the post-D7 Serenity drain strategy.
+
+Last updated: 2026-05-26
+
+## Goal
+
+Move the majority of remaining Docker workload off Serenity without recreating NFS/path-locality pain, while preserving the few roles that still legitimately need to live there until later infrastructure work is complete.
+
+This plan assumes:
+- qBittorrent and the torrent/media-locality lane stay on Serenity for now
+- PD is resource-constrained for a while, especially on RAM
+- `Wizarr` should be retired, not migrated
+- `Hawser` + `dockersocket` stay on Serenity because Dockhand uses them for remote management of Serenity's remaining Docker stacks
+- `Shelfmark` stays with the media-locality lane for now
+- Serenity's backup-DNS replacement should likely land on a small dedicated Raspberry Pi 4B rather than collapsing onto PD
+
+## Executive summary
+
+It is feasible to move the majority of Serenity's Docker workload off the box, but not as a single near-term wave.
+
+Right now the correct strategy is:
+1. retire obvious dead weight
+2. preserve the torrent/media-locality lane on Serenity
+3. keep Hawser/dockersocket, Newt, reranker, and backup-DNS as explicit temporary exceptions
+4. only move tiny low-risk apps early if PD headroom makes the move worth it
+5. do the real majority migration only after PD directly owns the relevant storage and path locality
+
+## Current placement decision matrix
+
+### Keep on Serenity now
+
+#### Media-locality lane
+- `GluetunVPN`
+- `qbit`
+- `qbit_manage`
+- `prowlarr`
+- `sonarr`
+- `sonarr-anime`
+- `radarr`
+- `lidarr`
+- `readarr`
+- `readarr-epub`
+- `bazarr`
+- `autobrr`
+- `unpackerr`
+- `shelfmark`
+
+Reason:
+- these either mount Serenity-owned `/mnt/user/data` directly or depend on workflows whose correctness currently assumes Serenity-local path ownership
+- splitting them early would turn this into an NFS-path rewrite project instead of a cleanup project
+
+#### Temporary intentional exceptions
+- `Hawser`
+- `dockersocket`
+- `Newt`
+- `technitium-dns-pilot`
+- `reranker`
+- `netdata`
+
+Reason:
+- `Hawser` + `dockersocket` remain part of the current Dockhand remote-management path
+- `Newt` still matters because Pangolin tunnels Serenity resources through Serenity-local Newt
+- `technitium-dns-pilot` preserves an off-PD DNS failure domain until replacement exists
+- `reranker` is not urgent to move while PD remains constrained
+- `netdata` is host-local monitoring; evaluate whether to simplify it later, not as a forced migration target now
+
+### Retire
+- `Wizarr`
+
+Reason:
+- operator confirmed it is unused and disposable
+- if onboarding needs return later, it can be rebuilt cleanly instead of migrated
+
+### Optional tiny move candidate
+- `Notifiarr`
+
+Reason:
+- low storage-locality coupling
+- but not worth forcing while PD remains constrained unless there is a concrete benefit
+
+## What should not be added to Serenity
+
+Default rule: add nothing new.
+
+Serenity is in a shrinking transitional role. New durable app responsibilities should go elsewhere unless they are explicitly temporary, host-local, and part of a migration or retirement aid.
+
+## Suggested long-term end state
+
+### PD long-term
+After storage cutover and capacity improvement, PD should absorb:
+- qBittorrent + Gluetun + qbit_manage
+- ARR family
+- `autobrr`
+- `unpackerr`
+- `shelfmark`
+- optional `Notifiarr`
+- `reranker` if PD remains the AI/control-plane center
+
+### Off-PD DNS resilience
+When Serenity retires, preserve at least one non-PD Technitium lane. Likely target:
+- dedicated Raspberry Pi 4B backup resolver
+
+### Serenity end state
+- no intentional durable production app role
+- no unique production dependency path left behind
+- host eligible for retirement after cooldown and verification
+
+## Execution-style Kanban board
+
+Use this as the working card map.
+
+### Epic A — Immediate cleanup and intention-locking
+
+#### A1. Retire Wizarr
+Status: ready
+Depends on: none
+
+Acceptance criteria:
+- `Wizarr` is removed from live Serenity runtime if still present
+- no reverse-proxy or bookmark expectation still points at it as a live service
+- docs no longer describe it as a migration target
+- rollback expectation is explicitly "rebuild if needed later," not "preserve migrated state"
+
+#### A2. Document Hawser + dockersocket as protected keepers
+Status: ready
+Depends on: none
+
+Acceptance criteria:
+- docs explicitly state Dockhand depends on `Hawser` + `dockersocket`
+- future cleanup cards do not treat them as accidental leftovers
+- any later rehome/removal must be paired with a replacement Dockhand management path
+
+#### A3. Decide whether netdata remains worth keeping
+Status: ready
+Depends on: none
+
+Acceptance criteria:
+- decision recorded as one of:
+  - keep as host-local monitoring until retirement
+  - simplify/replace with a lighter local signal
+  - remove after confirming broader monitoring already covers the operator need
+- if changed, verification includes equivalent host visibility from the replacement path
+
+### Epic B — Preserve Serenity's temporary intentional roles
+
+#### B1. Keep the media-locality lane stable on Serenity
+Status: locked
+Depends on: none
+
+Acceptance criteria:
+- no partial migration of qbit/ARR lane is attempted before storage design is ready
+- docs keep the lane grouped as an intentional temporary unit
+- future cards treat locality breakage as a rollback trigger, not a minor warning
+
+#### B2. Keep Hawser + dockersocket in place for Dockhand
+Status: locked
+Depends on: none
+
+Acceptance criteria:
+- `Hawser` and `dockersocket` remain healthy on Serenity
+- Dockhand remote management path still works for the remaining Serenity stacks
+- no proposal to move them is executed without a replacement management path
+
+#### B3. Keep Serenity-local Newt until Pangolin redesign exists
+Status: locked
+Depends on: none
+
+Acceptance criteria:
+- Pangolin-routed Serenity resources continue working
+- no retirement of Serenity-local Newt is attempted early
+- redesign work is handled as its own lane, not hidden inside app migration cards
+
+#### B4. Keep backup Technitium role on Serenity until replacement exists
+Status: locked
+Depends on: none
+
+Acceptance criteria:
+- off-PD DNS failure domain remains intact
+- no cutover collapses this role onto PD alone
+- replacement host is defined and verified before Serenity DNS role is removed
+
+#### B5. Keep reranker on Serenity until PD capacity or architecture changes
+Status: locked
+Depends on: none
+
+Acceptance criteria:
+- reranker remains stable where it is
+- no move is attempted just for neatness
+- any future move explicitly handles the live `/mnt/user/appdate/reranker` path oddity
+
+### Epic C — Optional tiny near-term moves
+
+#### C1. Re-evaluate Notifiarr against live PD headroom
+Status: ready
+Depends on: none
+
+Acceptance criteria:
+- current PD RAM/storage headroom is checked at execution time
+- clear recommendation recorded: move now, defer, or drop the idea
+- if moved, verification proves app health plus any expected integrations still work
+
+### Epic D — Replace Serenity special-case roles
+
+#### D1. Design the post-Serenity backup DNS lane
+Status: planned
+Depends on: none
+
+Target direction:
+- dedicated Raspberry Pi 4B backup resolver
+
+Acceptance criteria:
+- target host chosen and documented
+- role is explicitly outside PD's failure domain
+- expected sync, secrets, and verification model are documented before cutover
+
+#### D2. Build and validate the Pi 4B backup resolver
+Status: blocked
+Depends on: D1
+
+Acceptance criteria:
+- Pi 4B resolver is online and documented
+- sync path is defined
+- LAN clients can resolve through it as expected
+- it is clearly not dependent on PD for local authoritative continuity
+
+#### D3. Redesign Pangolin/Newt dependency
+Status: planned
+Depends on: none
+
+Acceptance criteria:
+- Serenity-hosted access path no longer requires Serenity-local Newt
+- replacement routing model is documented
+- cutover plan includes rollback and validation steps
+
+#### D4. Re-evaluate reranker final home
+Status: planned
+Depends on: none
+
+Acceptance criteria:
+- host choice made based on live resource reality and AI architecture, not symmetry
+- migration plan explicitly preserves existing data path and naming oddity handling
+
+### Epic E — Majority migration after storage ownership changes
+
+#### E1. Define PD storage cutover model
+Status: blocked
+Depends on: none
+
+Acceptance criteria:
+- exact target datasets/paths on PD are documented
+- ownership, mount semantics, and performance assumptions are explicit
+- migration no longer depends on NFS-style cross-host path fakery
+
+#### E2. Validate media/torrent path behavior on the target model
+Status: blocked
+Depends on: E1
+
+Acceptance criteria:
+- download path behavior validated
+- import path behavior validated
+- hardlink or equivalent behavior validated
+- post-processing behavior validated
+- ARR/qbit path mapping is internally consistent
+
+#### E3. Move qbit + Gluetun + qbit_manage
+Status: blocked
+Depends on: E2
+
+Acceptance criteria:
+- downloader lane works on the target without Serenity path dependence
+- VPN and management behavior are verified
+- rollback window is defined before source teardown
+
+#### E4. Move ARR + helper lane
+Status: blocked
+Depends on: E3
+
+Includes:
+- `prowlarr`
+- `sonarr`
+- `sonarr-anime`
+- `radarr`
+- `lidarr`
+- `readarr`
+- `readarr-epub`
+- `bazarr`
+- `autobrr`
+- `unpackerr`
+- `shelfmark`
+- optional `Notifiarr`
+
+Acceptance criteria:
+- libraries remain visible
+- imports and automation continue working
+- no stale Serenity-local path assumptions remain in configs
+
+#### E5. Post-cutover soak and verification
+Status: blocked
+Depends on: E4
+
+Acceptance criteria:
+- successful download/import verification exists
+- operator-facing health paths are normal
+- rollback confidence window passes without hidden path regressions
+
+### Epic F — Final Serenity retirement
+
+#### F1. Remove remaining special roles from Serenity
+Status: blocked
+Depends on: D2, D3, D4, E5
+
+Acceptance criteria:
+- no production path still depends on Serenity DNS role, Serenity-local Newt, or Serenity reranker placement
+- any retained backups are documented and off the critical path
+
+#### F2. Retire Serenity
+Status: blocked
+Depends on: F1
+
+Acceptance criteria:
+- no intentional production app remains
+- no unique resolver or tunnel role remains
+- docs describe Serenity as retired rather than transitional
+
+## Suggested execution order
+
+If only a few cards should move soon, use this order:
+1. A1 — Retire Wizarr
+2. A2 — Lock Hawser/dockersocket as intentional keepers
+3. A3 — Decide netdata end-state
+4. C1 — Re-evaluate Notifiarr only if there is real benefit
+5. D1/D2 — Create the replacement backup-DNS lane on Pi 4B
+6. D3/D4 — redesign special cases
+7. E1-E5 — execute the true majority migration only after PD storage ownership changes
+8. F1/F2 — retire Serenity
+
+## Recommendation
+
+Do not force a near-term "move the majority now" project.
+
+The correct near-term board is:
+- retire dead weight
+- keep the locality lane intact
+- keep Dockhand's Hawser path intact
+- build the future DNS replacement lane
+- wait for the storage and capacity conditions that make the real majority move sane