fizzlepoof/truenas-stacks
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
c2c71cfe577889ff41f86384c55931c1db37d85f
truenas-stacks/docs/operations/SECRETS_MANAGEMENT.md

7.7 KiB
Raw Blame History

Secrets Management

Overview

Secrets are managed in two layers:

  1. Live secrets — .env files on PD at /mnt/docker-ssd/docker/compose/<stack>/.env, gitignored and never committed to the main repo
  2. Encrypted backup — all .env files are synced to a private git-crypt encrypted Gitea repo at /mnt/docker-ssd/docker/secrets/

The sync is a one-command operation and is safe to re-run at any time.

Principles

  • .env files are gitignored in the main repo — never committed there
  • .env.example files with placeholder values are committed instead
  • The secrets repo encrypts everything at rest using git-crypt (symmetric key)
  • Secrets are generated with standard tools, never reused across services

Generating Secrets

# Database passwords
openssl rand -hex 24

# JWT / session secrets
openssl rand -hex 32

# API keys (where format allows)
openssl rand -base64 32

Live .env File Locations

All .env files live alongside their docker-compose.yaml:

/mnt/docker-ssd/docker/compose/
  ai/.env
  automation/.env
  databases/.env
  dev/.env
  documents/.env
  home/.env
  infrastructure/.env
  media/.env
  media/kima-hub/.env
  mesh-mqtt-observer/.env
  meshtastic/.env
  monitoring/.env
  photos/.env

Secrets Repo (git-crypt)
Gitea repo https://gitea.paccoco.com/fizzlepoof/homelab-secrets (private)
SSH remote ssh://git@10.5.30.6:2222/fizzlepoof/homelab-secrets.git
Local path on PD /mnt/docker-ssd/docker/secrets/
Symmetric key location /mnt/docker-ssd/docker/secrets/.git-crypt-secrets.key (if copied there) or retrieve from password manager
Key backup Password manager + C:\Users\Fizzlepoof\Downloads\.git-crypt-secrets.key

Repo structure

Directory Contents
env/ .env files for each stack, named <stack>.env
keys/ API keys and tokens
certs/ TLS certificates
tokens/ Service tokens (Paperless, LiteLLM, etc.)
ssh/ SSH private keys

All files except README.md and .gitattributes are encrypted by git-crypt on commit.

Syncing .env Files to the Secrets Repo

Use the sync script — no root required, safe to re-run:

bash /mnt/docker-ssd/docker/compose/scripts/sync-envs-to-secrets.sh

What it does:

  • Scans all stack directories under /mnt/docker-ssd/docker/compose/ for .env files
  • Copies changed/new files to /mnt/docker-ssd/docker/secrets/env/ as <stack>.env
  • Skips unchanged files
  • Commits and pushes to Gitea automatically

Run this any time you create or update a stack's .env file.

Operational rule: after any live .env change on PD, Doris or the operator must run the sync script and confirm whether it committed/pushed changes or reported the secrets repo already up to date. Do not treat live .env edits as complete until this sync step is checked.

git-crypt on TrueNAS

apt is blocked on TrueNAS Scale. git-crypt is installed via Docker.

Preferred installation location

/mnt/docker-ssd/bin/git-crypt

Compatibility note

The original bootstrap script installs to /root/bin/git-crypt, while the newer runbook expects /mnt/docker-ssd/bin/git-crypt. Standardize on /mnt/docker-ssd/bin/git-crypt and make sure PATH includes that directory for both root and truenas_admin.

Recommended PATH entry:

export PATH="/mnt/docker-ssd/bin:$PATH"

Re-installing after a full rebuild

Option 1 — one-liner:

sudo -i
docker run --rm -v /tmp:/out debian:bookworm-slim \
  bash -c "apt-get update -qq && apt-get install -y -qq git-crypt && cp /usr/bin/git-crypt /out/git-crypt"
mkdir -p /mnt/docker-ssd/bin
cp /tmp/git-crypt /mnt/docker-ssd/bin/git-crypt
chmod +x /mnt/docker-ssd/bin/git-crypt
echo 'export PATH="/mnt/docker-ssd/bin:$PATH"' >> /root/.bashrc
echo 'export PATH="/mnt/docker-ssd/bin:$PATH"' >> /home/truenas_admin/.bashrc

Option 2 — via dev stack compose:

cd /mnt/docker-ssd/docker/compose/dev
sudo docker compose --profile setup up git-crypt-init
# copies git-crypt to /root/bin — then move to persistent location:
sudo cp /root/bin/git-crypt /mnt/docker-ssd/bin/git-crypt

Unlocking the Secrets Repo on a New Machine

# Install git-crypt first (see above), then:
git clone ssh://git@10.5.30.6:2222/fizzlepoof/homelab-secrets.git /mnt/docker-ssd/docker/secrets
cd /mnt/docker-ssd/docker/secrets
git-crypt unlock /path/to/.git-crypt-secrets.key

After unlock, the env/ directory contains plaintext .env files ready to copy back into the stack directories.

Full Rebuild Runbook

On a fresh PD after reinstalling TrueNAS:

  1. Install git-crypt (Option 1 above — docker is available immediately after TrueNAS install)
  2. Restore secrets repo: 
    git clone ssh://git@10.5.30.6:2222/fizzlepoof/homelab-secrets.git /mnt/docker-ssd/docker/secrets
cd /mnt/docker-ssd/docker/secrets
git-crypt unlock /path/to/.git-crypt-secrets.key
  3. Restore .env files from secrets/env/ back to their stack directories: 
    cp /mnt/docker-ssd/docker/secrets/env/ai.env /mnt/docker-ssd/docker/compose/ai/.env
cp /mnt/docker-ssd/docker/secrets/env/databases.env /mnt/docker-ssd/docker/compose/databases/.env
# ... etc for each stack
  4. Clone the main stacks repo: 
    git clone ssh://git@10.5.30.6:2222/fizzlepoof/truenas-stacks.git /mnt/docker-ssd/docker/compose
  5. Redeploy stacks per DEPLOYMENT_CHECKLIST.md
  6. After any future live .env edit, immediately run: 
    export PATH="/mnt/docker-ssd/bin:$PATH"
bash /mnt/docker-ssd/docker/compose/scripts/sync-envs-to-secrets.sh
    Then verify the secrets repo is either updated and pushed or already unchanged.

Known Credentials Locations

Service File
shared-postgres databases/.env
shared-mariadb databases/.env
OpenWebUI DB ai/.env — user: openwebui, db: openwebui
Donetick home/.env + selfhosted.yaml
Meshmonitor meshtastic/.env
Gitea dev/.env
LiteLLM ai/.env
Paperless documents/.env
UniFi Doris operator automation/.env

Controller Export / Baseline Artifact Rules

Raw infrastructure exports must be treated as potentially secret-bearing even when they are not .env files.

Examples:

  • UniFi networkconf / portconf dumps
  • firewall/router/controller JSON exports
  • VPN client or server config exports
  • diagnostic snapshots taken from appliances or operator APIs

Permanent rules

  • Do not commit raw controller or appliance exports to the main repo.
  • Commit only sanitized markdown summaries or explicitly redacted artifacts.
  • If raw retention is needed, store the export in the encrypted secrets repo or another operator-only location outside the main repo.
  • Before committing infra artifacts, scan for obvious secret-bearing keys such as:
    • private_key
    • wireguard
    • token
    • secret
    • password
    • Authorization
    • cookie
    • client_secret

Incident reference

A real leak occurred on 2026-05-22 when a raw UniFi baseline export under home/doris-dashboard/docs/baselines/ was committed with embedded WireGuard secrets.

See:

  • docs/operations/INCIDENT_2026-05-22_UNIFI_BASELINE_SECRET_LEAK.md
  • home/doris-dashboard/docs/baselines/README.md

Security Reminders

  • Key backup: C:\Users\Fizzlepoof\Downloads\.git-crypt-secrets.key — also store in password manager
  • Regenerate Wings token on N.O.M.A.D. (was exposed in chat — TODO)
  • Regenerate N.O.M.A.D. Newt secret (was exposed in chat — TODO)
  • Disable signups in OpenWebUI after creating initial account
  • The Gitea token in setup-secrets-repo.sh should be rotated after initial setup
Reference in New Issue View Git Blame Copy Permalink