chore: sync homelab ops, identity, and monitoring docs
This commit is contained in:
@@ -33,6 +33,10 @@ automation/bin/karakeep_api.py /api/health --raw-path --pretty
|
||||
- `bin/pd_backup_postgres.sh` — creates gzipped `pg_dump` exports for the configured Postgres DB list (quote space-separated DB names in `.env`, e.g. `POSTGRES_DB_LIST="n8n paperless"`)
|
||||
- `bin/pd_backup_sync_to_serenity.sh` — `rsync`s configured backup/config roots to Serenity
|
||||
- `bin/run_pd_backups.sh` — wrapper that runs both steps in order
|
||||
- `bin/pd_restore_verify_postgres.sh` — restores the latest dump for each configured database into a throwaway Postgres container and fails on SQL errors
|
||||
- `bin/pd_restore_verify_appdata.sh` — stages one appdata/config tree back from Serenity and verifies required files exist
|
||||
- `bin/run_pd_restore_verification.sh` — wrapper that runs both restore checks in order
|
||||
- writes Prometheus textfile metrics for run status, duration, and last successful verification timestamp
|
||||
|
||||
These scripts are the live PD backup runner model.
|
||||
|
||||
@@ -49,6 +53,7 @@ Current documented live state:
|
||||
- the PD → Serenity backup flow is the intended deployed model
|
||||
- root cron on PD is the preferred scheduler
|
||||
- first-run verification and the deployed status are tracked in `docs/operations/PD_BACKUP_DEPLOYMENT.md` and `docs/planning/TODO.md`
|
||||
- quarterly restore verification is now expected via `bin/run_pd_restore_verification.sh`
|
||||
|
||||
## PD / TrueNAS deployment recommendation
|
||||
|
||||
@@ -93,6 +98,34 @@ Install in **root's crontab on PD**:
|
||||
# END PD BACKUPS
|
||||
```
|
||||
|
||||
## Recommended quarterly restore verification
|
||||
|
||||
Run from PD after a backup has completed:
|
||||
|
||||
```bash
|
||||
cd /mnt/docker-ssd/docker/compose/automation
|
||||
/usr/bin/bash bin/run_pd_restore_verification.sh
|
||||
```
|
||||
|
||||
This validates:
|
||||
|
||||
- latest configured Postgres dumps can be restored cleanly
|
||||
- a real config/appdata sample can be pulled back from Serenity
|
||||
- expected files still exist in the restored copy
|
||||
|
||||
If node-exporter is configured with the textfile collector, the wrapper also publishes:
|
||||
|
||||
- `pd_restore_verification_success`
|
||||
- `pd_restore_verification_last_run_timestamp_seconds`
|
||||
- `pd_restore_verification_last_success_timestamp_seconds`
|
||||
- `pd_restore_verification_duration_seconds`
|
||||
|
||||
Recommended root cron entry on PD:
|
||||
|
||||
```cron
|
||||
30 3 1 */3 * cd /mnt/docker-ssd/docker/compose/automation && /usr/bin/bash bin/run_pd_restore_verification.sh >> /mnt/tank/docker/backups/pd-restore-verify.log 2>&1
|
||||
```
|
||||
|
||||
Recommended first-run checklist:
|
||||
|
||||
```bash
|
||||
|
||||
Reference in New Issue
Block a user