Merge branch 'main' of gitea.paccoco.com:fizzlepoof/truenas-stacks

- scripts/setup-secrets-repo.sh: replace dpkg-deb install with Docker method
- dev/docker-compose.yaml: add git-crypt-init service (profile: setup)
- docs/operations/SECRETS_MANAGEMENT.md: document git-crypt encrypted repo, unlock procedure, rebuild runbook, and TrueNAS install methods

dev/docker-compose.yaml

@@ -39,6 +39,25 @@ services:
       retries: 3
       start_period: 30s
 
+  # Run once to install git-crypt on the TrueNAS host (apt is blocked there).
+  # Usage: docker compose --profile setup up git-crypt-init
+  git-crypt-init:
+    container_name: git-crypt-init
+    image: debian:bookworm-slim
+    profiles:
+      - setup
+    restart: "no"
+    volumes:
+      - /root/bin:/out
+    command: >
+      bash -c "
+        apt-get update -qq &&
+        apt-get install -y -qq git-crypt &&
+        cp /usr/bin/git-crypt /out/git-crypt &&
+        chmod +x /out/git-crypt &&
+        echo 'git-crypt installed to /root/bin/git-crypt'
+      "
+
   rackpeek:
     container_name: ix-rackpeek-rackpeek-1
     image: aptacode/rackpeek:latest

docs/operations/SECRETS_MANAGEMENT.md

@@ -2,8 +2,8 @@
 
 ## Principles
 - `.env` files are gitignored and never committed to any public repo
-- `.env.example` files are committed with placeholder values
 - Real `.env` files live only on the host at the stack directory
+- `.env.example` files are committed with placeholder values
 - Secrets are generated with standard tools, never reused across services
 
 ## Generating Secrets
@@ -25,27 +25,93 @@ All `.env` files live alongside their `docker-compose.yaml`:
 /mnt/docker-ssd/docker/compose/<stack>/.env
 ```
 
-## Backing Up .env Files
+---
+
+## Secrets Repo (git-crypt)
+
+All secrets are backed up to a **private, git-crypt encrypted** Gitea repo:
+
+| | |
+|---|---|
+| **Repo** | `https://gitea.paccoco.com/fizzlepoof/homelab-secrets` (private) |
+| **Local path** | `/mnt/docker-ssd/docker/secrets/` |
+| **Symmetric key** | `/root/.git-crypt-secrets.key` on PD |
+| **Key backup** | Password manager / USB — **do not lose this** |
+
+### Directory structure
+
+| Directory | Contents |
+|-----------|----------|
+| `env/` | `.env` files for each stack |
+| `keys/` | API keys and tokens |
+| `certs/` | TLS certificates |
+| `tokens/` | Service tokens (Paperless, LiteLLM, etc.) |
+| `ssh/` | SSH private keys |
+
+### Adding a secret
 
-### Short term: Private Gitea repo
-Create a private repo on Gitea (not GitHub) and push all .env files there:
 ```bash
-# In a separate directory, not the main stacks repo
+# As root on PD:
-git init homelab-secrets
+cp /mnt/docker-ssd/docker/compose/ai/.env /mnt/docker-ssd/docker/secrets/env/ai.env
+cd /mnt/docker-ssd/docker/secrets
+git add -A
+git -c user.name="Fizzlepoof" -c user.email="admin@paccoco.com" commit -m "chore: add ai stack env"
+git push
+```
+
+Files are encrypted automatically by git-crypt on push — no extra steps needed.
+
+### Unlocking on a new machine
+
+```bash
+git clone https://gitea.paccoco.com/fizzlepoof/homelab-secrets
 cd homelab-secrets
-# Copy and add .env files
+git-crypt unlock /path/to/.git-crypt-secrets.key
-git remote add origin https://gitea.paccoco.com/fizzlepoof/homelab-secrets.git
-git push -u origin main
 ```
 
-### Long term: Restic + offsite
+---
-Include the compose directory (with .env files) in a restic backup:
+
+## Installing git-crypt on TrueNAS (apt is blocked)
+
+### Option 1 — one-liner (as root)
 ```bash
-restic -r <repo> backup /mnt/docker-ssd/docker/compose --exclude="*.example"
+docker run --rm -v /tmp:/out debian:bookworm-slim \
+  bash -c "apt-get update -qq && apt-get install -y -qq git-crypt && cp /usr/bin/git-crypt /out/git-crypt"
+mkdir -p /root/bin && cp /tmp/git-crypt /root/bin/git-crypt && chmod +x /root/bin/git-crypt
+export PATH="/root/bin:$PATH"
 ```
 
-### Best long term: Vaultwarden
+### Option 2 — via Gitea stack compose (recommended for rebuilds)
-Once deployed, store each service's .env as a secure note in Vaultwarden. Single source of truth, accessible from anywhere.
+```bash
+cd /mnt/docker-ssd/docker/compose/dev
+docker compose --profile setup up git-crypt-init
+export PATH="/root/bin:$PATH"
+```
+
+The `git-crypt-init` service in `dev/docker-compose.yaml` handles the install automatically.
+
+### Make PATH persistent (add to /root/.bashrc)
+```bash
+echo 'export PATH="/root/bin:$PATH"' >> /root/.bashrc
+```
+
+---
+
+## Full Rebuild Runbook
+
+On a fresh PD:
+
+1. Install git-crypt (Option 1 or 2 above)
+2. Clone and unlock the secrets repo:
+   ```bash
+   git clone https://gitea.paccoco.com/fizzlepoof/homelab-secrets /mnt/docker-ssd/docker/secrets
+   cd /mnt/docker-ssd/docker/secrets
+   git-crypt unlock /path/to/.git-crypt-secrets.key
+   ```
+3. Copy `.env` files back into their stack directories from `secrets/env/`
+4. Redeploy stacks
+
+---
 
 ## Known Credentials Locations
 
@@ -62,3 +128,4 @@ Once deployed, store each service's .env as a secure note in Vaultwarden. Single
 - Regenerate Wings token on N.O.M.A.D. (was exposed in chat — TODO)
 - Regenerate N.O.M.A.D. Newt secret (was exposed in chat — TODO)
 - Disable signups in OpenWebUI after creating initial account
+- Back up `/root/.git-crypt-secrets.key` to password manager

scripts/setup-secrets-repo.sh

@@ -0,0 +1,163 @@
+#!/usr/bin/env bash
+# setup-secrets-repo.sh
+# One-shot: creates a private encrypted secrets repo on Gitea using git-crypt.
+# Run as root on PD.
+set -euo pipefail
+
+# ── CONFIG ────────────────────────────────────────────────────────────────────
+GITEA_HTTP_URL="http://10.5.1.6:3000"            # ← local Gitea HTTP port (check TrueNAS app config)
+GITEA_SSH_PORT="2222"                             # ← Gitea SSH port
+GITEA_USER="fizzlepoof"
+GITEA_TOKEN="aa711a194bd30f6355a0f5d396520d08762483aa"  # ← regenerate with write:repository + write:user scopes
+REPO_NAME="homelab-secrets"
+LOCAL_PATH="/mnt/docker-ssd/docker/secrets"
+KEY_EXPORT_PATH="/root/.git-crypt-secrets.key"   # back this up somewhere safe!
+# ─────────────────────────────────────────────────────────────────────────────
+
+RED='\033[0;31m'; GREEN='\033[0;32m'; YELLOW='\033[1;33m'; NC='\033[0m'
+info()    { echo -e "${GREEN}[+]${NC} $*"; }
+warn()    { echo -e "${YELLOW}[!]${NC} $*"; }
+die()     { echo -e "${RED}[✗]${NC} $*" >&2; exit 1; }
+
+[[ -z "$GITEA_TOKEN" ]] && die "GITEA_TOKEN is not set. Edit the CONFIG section and re-run."
+
+# ── 1. Dependencies ───────────────────────────────────────────────────────────
+info "Checking dependencies..."
+command -v git  &>/dev/null || die "git is not installed"
+command -v curl &>/dev/null || die "curl is not installed"
+
+if ! command -v git-crypt &>/dev/null; then
+  info "git-crypt not found — installing via Docker (apt is blocked on TrueNAS)..."
+  command -v docker &>/dev/null || die "docker is not installed"
+  docker run --rm -v /tmp:/out debian:bookworm-slim \
+    bash -c "apt-get update -qq && apt-get install -y -qq git-crypt && cp /usr/bin/git-crypt /out/git-crypt" \
+    || die "Docker-based git-crypt install failed"
+  mkdir -p /root/bin
+  cp /tmp/git-crypt /root/bin/git-crypt
+  chmod +x /root/bin/git-crypt
+  export PATH="/root/bin:$PATH"
+  info "git-crypt installed to /root/bin/git-crypt"
+fi
+
+# ── 2. Create private repo on Gitea ──────────────────────────────────────────
+info "Creating private repo '${REPO_NAME}' on Gitea..."
+HTTP_CODE=$(curl -s -o /tmp/gitea-create.json -w "%{http_code}" \
+  -X POST "${GITEA_HTTP_URL}/api/v1/user/repos" \
+  -H "Authorization: token ${GITEA_TOKEN}" \
+  -H "Content-Type: application/json" \
+  -d "{
+    \"name\":        \"${REPO_NAME}\",
+    \"description\": \"Homelab secrets — git-crypt encrypted\",
+    \"private\":     true,
+    \"auto_init\":   false
+  }")
+
+if [[ "$HTTP_CODE" == "201" ]]; then
+  info "Repo created successfully."
+elif [[ "$HTTP_CODE" == "409" ]]; then
+  warn "Repo already exists on Gitea — continuing."
+else
+  cat /tmp/gitea-create.json
+  die "Gitea API returned HTTP ${HTTP_CODE}. Check token and URL."
+fi
+
+REPO_URL="ssh://git@10.5.1.6:${GITEA_SSH_PORT}/${GITEA_USER}/${REPO_NAME}.git"
+
+# ── 3. Init local repo ────────────────────────────────────────────────────────
+info "Setting up local repo at ${LOCAL_PATH}..."
+mkdir -p "$LOCAL_PATH"
+cd "$LOCAL_PATH"
+
+if [[ ! -d .git ]]; then
+  git init
+  git remote add origin "$REPO_URL"
+else
+  warn ".git already exists — skipping init."
+  git remote set-url origin "$REPO_URL" 2>/dev/null || true
+fi
+
+# ── 4. Init git-crypt ─────────────────────────────────────────────────────────
+if [[ ! -d .git-crypt ]]; then
+  info "Initialising git-crypt..."
+  git-crypt init
+else
+  warn "git-crypt already initialised — skipping."
+fi
+
+# ── 5. Export symmetric key ───────────────────────────────────────────────────
+info "Exporting symmetric key to ${KEY_EXPORT_PATH}..."
+git-crypt export-key "$KEY_EXPORT_PATH"
+chmod 600 "$KEY_EXPORT_PATH"
+warn "BACK THIS KEY UP. Without it you cannot decrypt the repo on another machine."
+warn "  scp root@10.5.1.6:${KEY_EXPORT_PATH} /safe/location/"
+
+# ── 6. .gitattributes — encrypt everything except meta files ─────────────────
+info "Writing .gitattributes..."
+cat > .gitattributes << 'EOF'
+# Encrypt everything by default
+* filter=git-crypt diff=git-crypt
+
+# These stay plaintext
+.gitattributes !filter !diff
+README.md      !filter !diff
+EOF
+
+# ── 7. Directory structure ────────────────────────────────────────────────────
+info "Creating directory structure..."
+mkdir -p env keys certs tokens ssh
+
+cat > README.md << EOF
+# homelab-secrets
+
+Private secrets store for the homelab. All files except this README and
+.gitattributes are encrypted at rest with git-crypt.
+
+## Unlock on a new machine
+
+\`\`\`bash
+git clone ${REPO_URL}
+cd ${REPO_NAME}
+git-crypt unlock /path/to/.git-crypt-secrets.key
+\`\`\`
+
+## Structure
+
+| Directory | Contents |
+|-----------|----------|
+| \`env/\`   | .env files for each stack |
+| \`keys/\`  | API keys and tokens |
+| \`certs/\` | TLS certificates |
+| \`tokens/\`| Service tokens (Paperless, LiteLLM, etc.) |
+| \`ssh/\`   | SSH private keys |
+
+## Key location
+
+The symmetric key is stored at \`${KEY_EXPORT_PATH}\` on PD.
+Back it up to a password manager or USB drive.
+EOF
+
+# Placeholder files so dirs aren't empty
+for dir in env keys certs tokens ssh; do
+  touch "${dir}/.gitkeep"
+done
+
+# ── 8. Initial commit and push ────────────────────────────────────────────────
+info "Committing and pushing..."
+git add -A
+git -c user.name="Fizzlepoof" -c user.email="admin@paccoco.com" \
+  commit -m "chore: init encrypted secrets repo"
+
+git push -u origin main 2>/dev/null || git push -u origin master 2>/dev/null || \
+  die "Push failed. Check that Gitea credentials are configured (git credential store)."
+
+echo ""
+echo -e "${GREEN}━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━${NC}"
+echo -e "${GREEN}  Done!${NC}"
+echo -e "${GREEN}━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━${NC}"
+echo ""
+echo "  Repo:  ${REPO_URL}"
+echo "  Local: ${LOCAL_PATH}"
+echo "  Key:   ${KEY_EXPORT_PATH}"
+echo ""
+echo -e "${YELLOW}  Next: back up the key, then start dropping .env files into ${LOCAL_PATH}/env/${NC}"
+echo ""