chore: sync homelab ops, identity, and monitoring docs

docs/operations/BACKUP_POLICY.md

@@ -45,6 +45,10 @@ Recommended live deployment model on PD: a plain **root cron** job invoking `/us
 - Test restores quarterly
 - Test at least one app DB restore and one full-config restore
 - Verify that restored stacks can start with current compose files
+- Repo-staged verification runner: automation/bin/run_pd_restore_verification.sh
+- Current default verification targets:
+  - latest dumps from POSTGRES_DB_LIST
+  - Serenity-backed tank-docker/appdata/grafana sample restore with required files present
 
 ## Operational Notes
 - Media libraries on Unraid are large and may not need the same backup frequency as configs/databases

docs/operations/GITEA_SSO_PREP.md

@@ -0,0 +1,64 @@
+# Gitea SSO Prep
+
+This is the staged cutover plan for moving the live PD Gitea instance to Authelia-backed sign-in without flipping it live yet.
+
+## Current live shape
+
+- stack path: `dev/`
+- public URL: `https://gitea.paccoco.com`
+- internal HTTP: `http://gitea:3000` on the `pangolin` network
+- shared Postgres backend already in use
+- Gitea version observed live on PD: `1.25.3`
+
+## Safe hardening before SSO
+
+- disable public self-registration
+- keep normal local admin login working until Authelia sign-in is proven
+- do not change `ROOT_URL` or SSH settings during the auth cutover
+
+## Authelia side
+
+Add a dedicated OIDC client for Gitea in the live Authelia `configuration.yml` when ready to test.
+
+Recommended values:
+
+- `client_id`: `gitea`
+- `client_name`: `Gitea`
+- `authorization_policy`: `one_factor`
+- `scopes`: `openid`, `profile`, `email`, `groups`
+- `grant_types`: `authorization_code`
+- `response_types`: `code`
+- `token_endpoint_auth_method`: `client_secret_basic`
+
+Redirect URI guidance:
+
+- use the callback URL shown by the Gitea authentication-source form
+- for standard Gitea OIDC sources this is typically under `/user/oauth2/<source-name>/callback`
+- validate the exact callback path against the live Gitea UI before committing the Authelia client entry
+
+## Gitea side
+
+In Gitea:
+
+1. Sign in as a local admin.
+2. Go to `Site Administration` -> `Authentication Sources`.
+3. Add a new source of type `OpenID Connect`.
+4. Use Authelia discovery if the form supports it, otherwise enter the manual endpoints from `https://auth.paccoco.com/.well-known/openid-configuration`.
+
+Recommended source values:
+
+- name: `Authelia`
+- scopes: `openid profile email groups`
+- required claim mapping priority: email first, then username/login
+- keep the source enabled only after confirming the callback URL matches the Authelia client entry
+
+## Rollout notes
+
+- keep local admin auth available for the first pass
+- test with one admin account first
+- after successful sign-in, decide whether to keep local auth as break-glass only
+- do not disable Pangolin auth in front of Gitea until Authelia login is actually working end-to-end
+
+## Related weak spot
+
+The live Gitea app.ini observed on PD had public self-registration enabled. That should stay off for this homelab unless there is a deliberate reason to reopen it.

docs/operations/IDENTITY_AND_SSO.md

@@ -0,0 +1,60 @@
+# Identity and SSO
+
+Authelia is the staged identity provider for the homelab.
+
+## Why Authelia
+
+- centralizes auth for the growing pile of web apps
+- works with shared Postgres/Redis already in the lab
+- supports OIDC for apps that speak it properly
+- avoids inventing separate local auth for every exposed service
+
+## Current scope
+
+- stack path: identity/
+- host target: PlausibleDeniability
+- public entrypoint: auth.paccoco.com
+- status: repo-staged, not assumed live until deployed on PD
+
+## Initial bootstrap flow
+
+1. Copy identity/.env.example to identity/.env and generate fresh secrets.
+2. Copy identity/authelia/configuration.yml.example to identity/authelia/configuration.yml.
+3. Render concrete secrets and passwords into identity/authelia/configuration.yml on PD; do not rely on runtime env interpolation inside the YAML.
+4. Copy identity/authelia/users_database.example.yml to identity/authelia/users_database.yml.
+5. Generate an Argon2 password hash with the Authelia image and replace the sample hash.
+6. Create the authelia Postgres database and user in the shared database stack.
+7. Set `AUTHELIA_SESSION_REDIS_PASSWORD` from the live shared Redis stack on PD.
+8. Add a Pangolin resource for auth.paccoco.com pointing at container authelia:9091.
+9. Validate with docker compose --env-file .env config and docker exec authelia authelia config validate --config /config/configuration.yml.
+
+## Rollout order
+
+This repo stage is the control-plane bootstrap, not the full client cutover.
+
+After Authelia is healthy on PD, wire OIDC or forward-auth clients one at a time.
+
+Recommended first targets:
+
+- Grafana
+- Gitea
+- any new public app before it gains yet another standalone login
+
+## Grafana first-cutover notes
+
+- Use Authelia OIDC as a Generic OAuth provider in Grafana.
+- Keep Grafana's local admin login enabled for the first rollout.
+- After OIDC is verified, disable Pangolin auth in front of Grafana to avoid double-auth.
+
+## Gitea prep notes
+
+- The live Gitea stack is the `dev/` compose on PD.
+- Disable public self-registration before or during the SSO prep pass.
+- Keep local admin login working for the first OIDC test.
+- Stage the exact Gitea auth-source values in `docs/operations/GITEA_SSO_PREP.md` before creating the live source.
+
+## Notes
+
+- File-backed users are intentional for the initial rollout; this is enough for the current household scale.
+- If identity sprawl grows later, migrate the authentication backend to LDAP instead of bolting extra auth stores onto apps ad hoc.
+- OIDC signing keys and per-app client secrets should be generated during deployment on PD, not committed to the repo.

docs/operations/PD_BACKUP_DEPLOYMENT.md

@@ -68,6 +68,12 @@ Preferred: install in **root's crontab** on PD.
 # END PD BACKUPS
 ```
 
+Quarterly restore verification:
+
+```cron
+30 3 1 */3 * cd /mnt/docker-ssd/docker/compose/automation && /usr/bin/bash bin/run_pd_restore_verification.sh >> /mnt/tank/docker/backups/pd-restore-verify.log 2>&1
+```
+
 ## Post-deploy checks
 
 ```bash
@@ -76,6 +82,22 @@ ls -lh /mnt/tank/docker/backups/db-dumps
 ssh root@10.5.1.5 'find /mnt/user/backups/plausible-deniability -maxdepth 2 -type d | sort | head -40'
 ```
 
+## Restore verification
+
+After the backup path is stable, run the staged restore verifier from PD:
+
+```bash
+cd /mnt/docker-ssd/docker/compose/automation
+/usr/bin/bash bin/run_pd_restore_verification.sh
+```
+
+What it proves:
+
+- the latest configured Postgres dumps can restore into a temporary container
+- a real appdata/config sample can be staged back from Serenity
+- expected files still exist inside the restored sample
+- the latest run state can be exported to Prometheus via node-exporter's textfile collector
+
 ## Important limitations
 
 - Do **not** use `systemctl restart docker` on PD.