# PD Backup Deployment

Deploy the PD → Serenity backup runner from the real compose path on PD:

```bash
/mnt/docker-ssd/docker/compose/automation
```

## Why this model

- PD is the compose/data host, so backups should originate there.
- TrueNAS SCALE is happier with plain cron + shell than extra appliance-fighting service glue.
- Root cron on PD avoids the common `sudo`/Docker socket permission mess.
- Backups should still run even if n8n is down.

## Prerequisites

1. Repo content present on PD at `/mnt/docker-ssd/docker/compose/automation`
2. Writable dump directory:
   ```bash
   mkdir -p /mnt/tank/docker/backups/db-dumps
   ```
3. SSH trust from PD to Serenity for the selected account
4. Real `.env` created from `.env.example`
5. Preferred: install the cron job in root's crontab on PD
6. If using a non-root PD account instead, `sudo -n /usr/bin/docker` must work

## Suggested `.env` values to review

```dotenv
SERENITY_BACKUP_HOST=root@10.5.30.5
SERENITY_BACKUP_ROOT=/mnt/user/backups/plausible-deniability
PD_APPDATA_ROOT=/mnt/docker-ssd/docker/appdata
PD_DATABASES_ROOT=/mnt/docker-ssd/docker/databases
PD_TANK_DOCKER_ROOT=/mnt/tank/docker
PD_DB_DUMP_ROOT=/mnt/tank/docker/backups/db-dumps
POSTGRES_CONTAINER=shared-postgres
POSTGRES_DB_LIST="gitea n8n paperless"
BACKUP_SSH_KEY=/home/truenas_admin/.ssh/serenity_backup_ed25519
DOCKER_BIN=/usr/bin/docker
SUDO_BIN=/usr/bin/sudo
USE_SUDO_FOR_DOCKER=true
```

## Manual first run

```bash
cd /mnt/docker-ssd/docker/compose/automation
cp .env.example .env
chmod 600 .env
mkdir -p /mnt/tank/docker/backups/db-dumps
/usr/bin/bash bin/run_pd_backups.sh
```

Confirm:

- fresh `gitea_*.sql.gz`, `n8n_*.sql.gz`, and `paperless_*.sql.gz` files appear in `/mnt/tank/docker/backups/db-dumps`
- appdata/database/tank-docker content lands under the chosen Serenity backup root
- no sudo prompt appeared during `docker exec`

## Cron install

Preferred: install in **root's crontab** on PD.

```cron
# BEGIN PD BACKUPS
15 2 * * * cd /mnt/docker-ssd/docker/compose/automation && /usr/bin/bash bin/run_pd_backups.sh >> /mnt/tank/docker/backups/pd-backups.log 2>&1
# END PD BACKUPS
```

Quarterly restore verification:

```cron
30 3 1 */3 * cd /mnt/docker-ssd/docker/compose/automation && /usr/bin/bash bin/run_pd_restore_verification.sh >> /mnt/tank/docker/backups/pd-restore-verify.log 2>&1
```

## Post-deploy checks

```bash
tail -100 /mnt/tank/docker/backups/pd-backups.log
ls -lh /mnt/tank/docker/backups/db-dumps
ssh root@10.5.30.5 'find /mnt/user/backups/plausible-deniability -maxdepth 2 -type d | sort | head -40'
```

## Restore verification

After the backup path is stable, run the staged restore verifier from PD:

```bash
cd /mnt/docker-ssd/docker/compose/automation
/usr/bin/bash bin/run_pd_restore_verification.sh
```

What it proves:

- the latest configured Postgres dumps can restore into a temporary container
- a real appdata/config sample can be staged back from Serenity
- expected files still exist inside the restored sample
- the latest run state can be exported to Prometheus via node-exporter's textfile collector

Current expectation: `POSTGRES_DB_LIST` includes `gitea` so the backup set contains a logical dump for the Gitea database in addition to the raw synced Postgres data directory.

## Important limitations

- Do **not** use `systemctl restart docker` on PD.
- Keep source/destination paths under `/mnt/...`.
- `rsync --delete` is intentional; use a dedicated destination root, not a shared miscellaneous folder.
- This scaffolding does not yet back up `.env` files here because those are already handled by the separate encrypted/private-repo process.
- If you insist on a non-root cron user on PD, you will need passwordless Docker access (`sudo -n /usr/bin/docker ...`) and write access to the chosen dump directory.
- For Gitea, do not treat the live-synced Postgres datadir as the main restore proof; keep and verify the `gitea` logical dump as part of the normal restore-verification cycle.