Publish sanitized current homelab architecture
Some checks failed
public-safety / privacy-and-secret-scan (push) Has been cancelled
Some checks failed
public-safety / privacy-and-secret-scan (push) Has been cancelled
This commit is contained in:
24
.github/workflows/public-safety.yml
vendored
Normal file
24
.github/workflows/public-safety.yml
vendored
Normal file
@@ -0,0 +1,24 @@
|
|||||||
|
name: public-safety
|
||||||
|
|
||||||
|
on:
|
||||||
|
push:
|
||||||
|
pull_request:
|
||||||
|
|
||||||
|
permissions:
|
||||||
|
contents: read
|
||||||
|
|
||||||
|
jobs:
|
||||||
|
privacy-and-secret-scan:
|
||||||
|
runs-on: ubuntu-latest
|
||||||
|
steps:
|
||||||
|
- uses: actions/checkout@v4
|
||||||
|
with:
|
||||||
|
fetch-depth: 0
|
||||||
|
|
||||||
|
- name: Check public-safe content
|
||||||
|
run: python3 scripts/check-public-safety.py
|
||||||
|
|
||||||
|
- name: Scan Git history for secrets
|
||||||
|
uses: gitleaks/gitleaks-action@v2
|
||||||
|
env:
|
||||||
|
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
||||||
19
.gitignore
vendored
Normal file
19
.gitignore
vendored
Normal file
@@ -0,0 +1,19 @@
|
|||||||
|
# Local runtime files
|
||||||
|
.env
|
||||||
|
.env.*
|
||||||
|
!.env.example
|
||||||
|
|
||||||
|
# Python
|
||||||
|
__pycache__/
|
||||||
|
*.py[cod]
|
||||||
|
.venv/
|
||||||
|
|
||||||
|
# Editors and operating systems
|
||||||
|
.idea/
|
||||||
|
.vscode/
|
||||||
|
.DS_Store
|
||||||
|
|
||||||
|
# Generated scans and local artifacts
|
||||||
|
*.sarif
|
||||||
|
*.log
|
||||||
|
reports/
|
||||||
21
LICENSE
Normal file
21
LICENSE
Normal file
@@ -0,0 +1,21 @@
|
|||||||
|
MIT License
|
||||||
|
|
||||||
|
Copyright (c) 2026 fizzlepoof
|
||||||
|
|
||||||
|
Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||||
|
of this software and associated documentation files (the "Software"), to deal
|
||||||
|
in the Software without restriction, including without limitation the rights
|
||||||
|
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||||
|
copies of the Software, and to permit persons to whom the Software is
|
||||||
|
furnished to do so, subject to the following conditions:
|
||||||
|
|
||||||
|
The above copyright notice and this permission notice shall be included in all
|
||||||
|
copies or substantial portions of the Software.
|
||||||
|
|
||||||
|
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||||
|
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||||
|
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||||
|
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||||
|
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||||
|
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
||||||
|
SOFTWARE.
|
||||||
88
README.md
Normal file
88
README.md
Normal file
@@ -0,0 +1,88 @@
|
|||||||
|
# Homelab Showcase
|
||||||
|
|
||||||
|
A public-safe overview of a real multi-host homelab built around segmented networking, self-hosted applications, resilient storage, local AI, offline services, and low-power edge nodes.
|
||||||
|
|
||||||
|
> This repository is intentionally **not** the operational source of truth. Exact addresses, domains, account names, device identifiers, locations, mount paths, credentials, and raw controller exports are omitted or replaced with examples.
|
||||||
|
|
||||||
|
## What this lab is designed to do
|
||||||
|
|
||||||
|
- Keep household services useful during an internet outage
|
||||||
|
- Separate trusted devices, servers, IoT, cameras, guests, and lab workloads
|
||||||
|
- Centralize applications without making one server responsible for every failure domain
|
||||||
|
- Keep bulk storage close to media ingestion and backup workflows
|
||||||
|
- Route local AI work to the most appropriate always-on or high-performance host
|
||||||
|
- Publish selected services through authenticated tunnels instead of broad inbound exposure
|
||||||
|
- Preserve recovery options with encrypted secrets, snapshots, versioned sync, and documented rollback steps
|
||||||
|
|
||||||
|
## Current architecture
|
||||||
|
|
||||||
|
| Node | Platform | Primary role |
|
||||||
|
|---|---|---|
|
||||||
|
| **Application host** | TrueNAS SCALE | Primary application, database, identity, observability, media, and shared AI host |
|
||||||
|
| **Storage host** | Storage-focused Linux server | Bulk storage, media-ingestion pipeline, synchronization hub, remote-support relay, and selected CPU services |
|
||||||
|
| **Resilience host** | Ubuntu Server | Offline knowledge, resilient local tools, home automation VM, game orchestration, and secondary AI/DNS services |
|
||||||
|
| **GPU workstation** | CachyOS workstation | Operator workstation and opportunistic high-performance GPU compute |
|
||||||
|
| **Edge node** | Low-power Linux node | Weather dispatch and mesh-radio edge services |
|
||||||
|
|
||||||
|
See [Architecture](docs/architecture.md) for the data flow and [Services](docs/services.md) for the current service placement.
|
||||||
|
|
||||||
|
## Design highlights
|
||||||
|
|
||||||
|
- **Segmented UniFi network:** management, trusted clients, servers, IoT, cameras, guests, and quarantine are separate policy lanes.
|
||||||
|
- **Authenticated ingress:** selected applications use a tunnel gateway, an internal reverse proxy, and SSO. Most services have no direct internet-facing listener.
|
||||||
|
- **Three-node DNS design:** one authoritative configuration source synchronizes to two backup resolvers; public fallback DNS is treated separately from private-zone continuity.
|
||||||
|
- **Shared databases:** PostgreSQL, MariaDB, and Redis provide a managed data layer for compatible application stacks.
|
||||||
|
- **Split storage and compute:** application compute is concentrated on the primary host while large media/photo datasets stay on the storage server.
|
||||||
|
- **Tiered local AI:** an always-on light tier handles routine work; the GPU workstation is optional capacity rather than a hard dependency.
|
||||||
|
- **Offline-first node:** documentation, educational content, notes, utilities, and selected models remain available locally.
|
||||||
|
- **Encrypted operations:** runtime secrets live outside application repositories and are encrypted before entering version control.
|
||||||
|
|
||||||
|
## Repository map
|
||||||
|
|
||||||
|
```text
|
||||||
|
docs/
|
||||||
|
architecture.md High-level topology and request flows
|
||||||
|
networking.md Segmentation and access-control model
|
||||||
|
operations.md Deployment, verification, backup, and rollback patterns
|
||||||
|
security.md Public/private boundary and secret-handling rules
|
||||||
|
services.md Current service placement by host
|
||||||
|
examples/
|
||||||
|
compose/ Sanitized Compose and environment examples
|
||||||
|
scripts/
|
||||||
|
check-public-safety.py
|
||||||
|
```
|
||||||
|
|
||||||
|
## Reusable patterns
|
||||||
|
|
||||||
|
- [Sanitized Compose example](examples/compose/docker-compose.yml)
|
||||||
|
- [Environment template](examples/compose/.env.example)
|
||||||
|
- [Public-safety scanner](scripts/check-public-safety.py)
|
||||||
|
- [Operational practices](docs/operations.md)
|
||||||
|
- [Security model](docs/security.md)
|
||||||
|
|
||||||
|
## Public-safety policy
|
||||||
|
|
||||||
|
Every commit is checked for:
|
||||||
|
|
||||||
|
- private or management addresses
|
||||||
|
- MAC addresses and device identifiers
|
||||||
|
- internal domains and personal filesystem paths
|
||||||
|
- private-key material, tokens, passwords, and authorization headers
|
||||||
|
- real `.env` files and key-bearing file types
|
||||||
|
|
||||||
|
Gitleaks also scans the complete Git history in CI. See [Security](docs/security.md).
|
||||||
|
|
||||||
|
## Scope
|
||||||
|
|
||||||
|
This repository demonstrates architecture and operating patterns. It deliberately excludes:
|
||||||
|
|
||||||
|
- deployable production secrets
|
||||||
|
- exact network coordinates
|
||||||
|
- raw exports, backups, logs, and incident artifacts
|
||||||
|
- household or personal records
|
||||||
|
- device serial numbers and persistent client identifiers
|
||||||
|
- live tunnel, VPN, DNS, or identity-provider configuration
|
||||||
|
|
||||||
|
## License
|
||||||
|
|
||||||
|
MIT — see [LICENSE](LICENSE).
|
||||||
17
SECURITY.md
Normal file
17
SECURITY.md
Normal file
@@ -0,0 +1,17 @@
|
|||||||
|
# Security Policy
|
||||||
|
|
||||||
|
## Scope
|
||||||
|
|
||||||
|
This repository contains public-safe architecture documentation and sanitized examples only. It does not accept production credentials, raw infrastructure exports, private topology, or personal data.
|
||||||
|
|
||||||
|
## Reporting a problem
|
||||||
|
|
||||||
|
If you find sensitive information in this repository:
|
||||||
|
|
||||||
|
1. Do not quote it in a public issue.
|
||||||
|
2. Contact the repository owner through an already established private channel.
|
||||||
|
3. Include only the affected path and the type of exposure until the value has been revoked.
|
||||||
|
|
||||||
|
A confirmed credential exposure is handled by rotating the credential first, removing it from current files, rewriting affected Git history, and verifying a fresh clone.
|
||||||
|
|
||||||
|
For the full repository policy, see [docs/security.md](docs/security.md).
|
||||||
141
docs/architecture.md
Normal file
141
docs/architecture.md
Normal file
@@ -0,0 +1,141 @@
|
|||||||
|
# Architecture
|
||||||
|
|
||||||
|
## Topology
|
||||||
|
|
||||||
|
```mermaid
|
||||||
|
flowchart TB
|
||||||
|
Internet((Internet))
|
||||||
|
Edge[Authenticated tunnel gateway]
|
||||||
|
SSO[Identity and SSO]
|
||||||
|
Proxy[Internal reverse proxy]
|
||||||
|
|
||||||
|
subgraph Home[Segmented home network]
|
||||||
|
subgraph Primary[Primary application host]
|
||||||
|
Apps[Household and productivity apps]
|
||||||
|
Data[Shared databases and caches]
|
||||||
|
Media[Media, photos, and documents]
|
||||||
|
Observe[Monitoring and dashboards]
|
||||||
|
AI[Always-on AI and search]
|
||||||
|
end
|
||||||
|
|
||||||
|
subgraph Storage[Storage and ingestion host]
|
||||||
|
Pool[Bulk ZFS storage]
|
||||||
|
Arr[Media automation and download pipeline]
|
||||||
|
Sync[File synchronization and backups]
|
||||||
|
Remote[Remote-support relay]
|
||||||
|
end
|
||||||
|
|
||||||
|
subgraph Resilience[Offline and resilience host]
|
||||||
|
Offline[Offline knowledge and education]
|
||||||
|
HomeAuto[Home automation VM]
|
||||||
|
Games[Game-server control plane]
|
||||||
|
Secondary[Backup DNS and local AI]
|
||||||
|
Agents[Automation and agent services]
|
||||||
|
end
|
||||||
|
|
||||||
|
Workstation[GPU operator workstation]
|
||||||
|
EdgeNode[Low-power weather and radio node]
|
||||||
|
end
|
||||||
|
|
||||||
|
Internet --> Edge --> Proxy
|
||||||
|
Proxy --> SSO
|
||||||
|
SSO --> Apps
|
||||||
|
Apps --> Data
|
||||||
|
Apps --> Pool
|
||||||
|
Media --> Pool
|
||||||
|
Arr --> Pool
|
||||||
|
Observe --> Apps
|
||||||
|
AI -. optional heavy jobs .-> Workstation
|
||||||
|
Secondary -. resilient local path .-> AI
|
||||||
|
Agents --> Apps
|
||||||
|
Sync --> Workstation
|
||||||
|
EdgeNode --> Agents
|
||||||
|
Remote -. private overlay .-> Workstation
|
||||||
|
```
|
||||||
|
|
||||||
|
## Host responsibilities
|
||||||
|
|
||||||
|
### Primary application host
|
||||||
|
|
||||||
|
The primary host carries the application plane:
|
||||||
|
|
||||||
|
- shared PostgreSQL, MariaDB, and Redis
|
||||||
|
- identity, SSO, ingress routing, and tunnel clients
|
||||||
|
- media front ends, photo management, and document management
|
||||||
|
- dashboards, monitoring, metrics, logs, and notifications
|
||||||
|
- household applications and workflow automation
|
||||||
|
- light-tier model serving, search, RAG, and vector storage
|
||||||
|
- self-hosted development and administration tools
|
||||||
|
|
||||||
|
The host is intentionally not the only copy of storage, DNS, or offline knowledge.
|
||||||
|
|
||||||
|
### Storage and ingestion host
|
||||||
|
|
||||||
|
The storage host owns the large-capacity data plane and workloads that benefit from data locality:
|
||||||
|
|
||||||
|
- ZFS-backed bulk datasets
|
||||||
|
- media acquisition, organization, and post-processing
|
||||||
|
- synchronized document storage with versioning
|
||||||
|
- backup targets and retained application data
|
||||||
|
- private remote-support rendezvous and relay
|
||||||
|
- selected CPU-only inference utilities
|
||||||
|
- a secondary DNS replica
|
||||||
|
|
||||||
|
### Offline and resilience host
|
||||||
|
|
||||||
|
The resilience host is designed to remain useful when public services are unavailable:
|
||||||
|
|
||||||
|
- offline reference libraries and educational content
|
||||||
|
- local notes and browser utilities
|
||||||
|
- local model and vector services
|
||||||
|
- agent memory and household helper applications
|
||||||
|
- Home Assistant running in a dedicated virtual machine
|
||||||
|
- game-server management
|
||||||
|
- mesh-radio observation
|
||||||
|
- a secondary DNS replica
|
||||||
|
|
||||||
|
### GPU workstation
|
||||||
|
|
||||||
|
The workstation provides high-performance GPU capacity for interactive or heavy jobs. Routine background tasks must continue when it is asleep or offline.
|
||||||
|
|
||||||
|
### Low-power edge node
|
||||||
|
|
||||||
|
The edge node handles always-on weather and mesh-radio functions close to the attached hardware. It reports upstream but is not part of the main application failure domain.
|
||||||
|
|
||||||
|
## Request flows
|
||||||
|
|
||||||
|
### External application request
|
||||||
|
|
||||||
|
```text
|
||||||
|
client → tunnel gateway → internal reverse proxy → SSO policy → application
|
||||||
|
```
|
||||||
|
|
||||||
|
The tunnel endpoint is the only public routing layer. Application containers do not require direct inbound router exposure.
|
||||||
|
|
||||||
|
### Internal application request
|
||||||
|
|
||||||
|
```text
|
||||||
|
trusted client → internal DNS → application or reverse proxy → service
|
||||||
|
```
|
||||||
|
|
||||||
|
### AI request
|
||||||
|
|
||||||
|
```text
|
||||||
|
application → model gateway → always-on local model
|
||||||
|
↘ optional GPU workstation
|
||||||
|
```
|
||||||
|
|
||||||
|
### Storage request
|
||||||
|
|
||||||
|
```text
|
||||||
|
application host → authenticated network storage mount → storage dataset
|
||||||
|
```
|
||||||
|
|
||||||
|
## Failure-domain rules
|
||||||
|
|
||||||
|
1. Losing the GPU workstation must not stop routine automation.
|
||||||
|
2. Losing public DNS must not be confused with losing the private DNS zone.
|
||||||
|
3. Losing the primary application host must not destroy bulk datasets or backups.
|
||||||
|
4. A tunnel or SSO failure must not expose an application directly.
|
||||||
|
5. The offline host must retain useful local content without WAN access.
|
||||||
|
6. Changes to storage, networking, identity, and databases require explicit rollback points.
|
||||||
69
docs/networking.md
Normal file
69
docs/networking.md
Normal file
@@ -0,0 +1,69 @@
|
|||||||
|
# Networking
|
||||||
|
|
||||||
|
## Segmentation model
|
||||||
|
|
||||||
|
The network uses purpose-specific policy lanes rather than one trusted broadcast domain.
|
||||||
|
|
||||||
|
| Lane | Intended occupants | Default posture |
|
||||||
|
|---|---|---|
|
||||||
|
| **Management** | Gateway, switches, access points, controllers | Administrative access only |
|
||||||
|
| **Trusted** | Human-operated laptops, phones, and workstations | May initiate approved internal connections |
|
||||||
|
| **Servers** | Application, storage, DNS, and automation hosts | Explicit service exposure |
|
||||||
|
| **IoT** | Appliances and embedded clients | Restricted from internal networks by default |
|
||||||
|
| **Cameras** | Video and security devices | More restrictive than general IoT |
|
||||||
|
| **Guest** | Visitors and temporary clients | Internet only |
|
||||||
|
| **Quarantine** | Legacy or unidentified devices | Minimal DNS, time, and internet access |
|
||||||
|
| **Lab** | Disposable security and development systems | Default deny toward production |
|
||||||
|
|
||||||
|
Exact VLAN identifiers, addressing, SSIDs, and device assignments are intentionally absent from this public repository.
|
||||||
|
|
||||||
|
## Policy principles
|
||||||
|
|
||||||
|
- Inter-lane access is denied unless a specific workflow requires it.
|
||||||
|
- Management interfaces are never reachable from guest, IoT, camera, or lab lanes.
|
||||||
|
- IoT exceptions are limited to named services and destinations.
|
||||||
|
- Discovery protocols are bridged only when a real use case requires them.
|
||||||
|
- DNS, DHCP, and time synchronization are treated as explicit dependencies.
|
||||||
|
- A trusted operator path stays available during every network change.
|
||||||
|
- Client migrations happen in small batches with rollback after each batch.
|
||||||
|
|
||||||
|
## DNS resilience
|
||||||
|
|
||||||
|
The lab uses three synchronized internal resolvers:
|
||||||
|
|
||||||
|
1. a primary configuration source on the application host
|
||||||
|
2. one replica on the offline/resilience host
|
||||||
|
3. one replica on the storage host
|
||||||
|
|
||||||
|
The replicas receive configuration on a schedule and are verified from another host. External recursive resolvers may provide public-name fallback, but they cannot replace authoritative answers for the private zone.
|
||||||
|
|
||||||
|
## Remote access
|
||||||
|
|
||||||
|
Two complementary mechanisms are used:
|
||||||
|
|
||||||
|
- **Authenticated application ingress:** tunnel gateway → reverse proxy → SSO → selected service
|
||||||
|
- **Private administrative overlay:** device-to-device access for operators and private support workflows
|
||||||
|
|
||||||
|
Most applications do not expose router-forwarded ports. Protocols that cannot use the HTTP tunnel are handled as narrow, documented exceptions.
|
||||||
|
|
||||||
|
## Container networking
|
||||||
|
|
||||||
|
Compose projects use named external networks for shared dependencies:
|
||||||
|
|
||||||
|
- a data network for databases and caches
|
||||||
|
- an ingress network for explicitly published applications
|
||||||
|
- optional service-family networks for AI and observability
|
||||||
|
|
||||||
|
Applications join only the networks they need. The reverse proxy does not receive unrestricted control of the container runtime.
|
||||||
|
|
||||||
|
## Validation gates
|
||||||
|
|
||||||
|
After a network change, verify:
|
||||||
|
|
||||||
|
1. gateway and controller access from a trusted client
|
||||||
|
2. DHCP lease and intended lane placement
|
||||||
|
3. private and public DNS answers
|
||||||
|
4. the exact application protocol, not only ping
|
||||||
|
5. cross-lane allow and deny behavior
|
||||||
|
6. recovery after client reassociation or host reboot
|
||||||
|
7. absence of broad temporary allow rules
|
||||||
78
docs/operations.md
Normal file
78
docs/operations.md
Normal file
@@ -0,0 +1,78 @@
|
|||||||
|
# Operations
|
||||||
|
|
||||||
|
## Deployment order
|
||||||
|
|
||||||
|
A predictable dependency order reduces partial deployments:
|
||||||
|
|
||||||
|
1. storage mounts and host prerequisites
|
||||||
|
2. shared databases and caches
|
||||||
|
3. ingress, identity, DNS, and management services
|
||||||
|
4. observability and notifications
|
||||||
|
5. application stacks
|
||||||
|
6. optional AI, media, and edge integrations
|
||||||
|
|
||||||
|
## Change workflow
|
||||||
|
|
||||||
|
1. **Read first.** Capture current host, service, network, and storage state.
|
||||||
|
2. **Define scope.** Name the exact stack, files, and expected result.
|
||||||
|
3. **Create rollback.** Back up configuration and state before mutation.
|
||||||
|
4. **Validate configuration.** Render Compose, parse structured files, and run syntax checks.
|
||||||
|
5. **Apply narrowly.** Restart or recreate only the service that consumes the change.
|
||||||
|
6. **Verify from the consumer side.** A process being `running` is not proof that the workflow works.
|
||||||
|
7. **Read back external state.** Confirm the controller, API, repository, or database actually retained the change.
|
||||||
|
8. **Document the result.** Record current truth, rollback location, and deferred work.
|
||||||
|
|
||||||
|
## Compose validation
|
||||||
|
|
||||||
|
A sanitized example stack is provided under `examples/compose/`.
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cp examples/compose/.env.example examples/compose/.env
|
||||||
|
# Replace placeholders only in the untracked .env file.
|
||||||
|
docker compose --env-file examples/compose/.env \
|
||||||
|
-f examples/compose/docker-compose.yml config
|
||||||
|
```
|
||||||
|
|
||||||
|
Never commit the generated `.env` file.
|
||||||
|
|
||||||
|
## Service verification
|
||||||
|
|
||||||
|
Use multiple layers of evidence:
|
||||||
|
|
||||||
|
- container or service state
|
||||||
|
- health endpoint
|
||||||
|
- recent logs
|
||||||
|
- dependency connectivity
|
||||||
|
- application-level request
|
||||||
|
- persistence after restart
|
||||||
|
- monitoring visibility
|
||||||
|
|
||||||
|
For stateful applications, also check a representative database object or file after restart.
|
||||||
|
|
||||||
|
## Backup model
|
||||||
|
|
||||||
|
- filesystem snapshots protect bulk datasets
|
||||||
|
- application-aware exports protect databases and stateful services
|
||||||
|
- encrypted Git stores recoverable configuration secrets
|
||||||
|
- normal Git stores source, sanitized templates, and operator documentation
|
||||||
|
- synchronized user files use versioning in addition to storage snapshots
|
||||||
|
- restore tests are scheduled; backup creation alone is not treated as recovery proof
|
||||||
|
|
||||||
|
## Storage and data locality
|
||||||
|
|
||||||
|
Large data stays on the storage host. Applications consume it over authenticated mounts where practical. Ingestion workloads remain close to the data when moving them would create avoidable network traffic or fragile cross-host dependencies.
|
||||||
|
|
||||||
|
## Monitoring and notifications
|
||||||
|
|
||||||
|
Metrics, logs, uptime checks, and push notifications are separate components so one dashboard failure does not erase the underlying evidence. Alerts should be low-volume, actionable, and routed to an operator channel with enough context to diagnose.
|
||||||
|
|
||||||
|
## Updating documentation
|
||||||
|
|
||||||
|
Documentation is divided into:
|
||||||
|
|
||||||
|
- **current state:** what is running and where
|
||||||
|
- **operating procedure:** how to change and verify it
|
||||||
|
- **historical evidence:** dated incident or migration records
|
||||||
|
- **future plan:** proposed work that is not yet live
|
||||||
|
|
||||||
|
Dated historical notes should never read like the active backlog. Public documentation describes roles and patterns; exact operational coordinates remain private.
|
||||||
67
docs/security.md
Normal file
67
docs/security.md
Normal file
@@ -0,0 +1,67 @@
|
|||||||
|
# Security
|
||||||
|
|
||||||
|
## Public/private boundary
|
||||||
|
|
||||||
|
This public repository contains architectural patterns, sanitized examples, and high-level service placement. It is not used for production deployment.
|
||||||
|
|
||||||
|
The private operational repository contains exact topology, runtime paths, deployment notes, and rollback artifacts. A separate encrypted repository stores environment files and other secrets.
|
||||||
|
|
||||||
|
## Never publish
|
||||||
|
|
||||||
|
- credentials, tokens, cookies, private keys, recovery codes, or session material
|
||||||
|
- real `.env` files
|
||||||
|
- internal or management addresses
|
||||||
|
- MAC addresses, serial numbers, client IDs, or tunnel IDs
|
||||||
|
- real domains, SSIDs, account names, or personal filesystem paths
|
||||||
|
- physical locations or household records
|
||||||
|
- raw controller exports, logs, database dumps, or backups
|
||||||
|
- screenshots containing dashboards, account menus, or browser sessions
|
||||||
|
|
||||||
|
## Secret workflow
|
||||||
|
|
||||||
|
1. Generate or receive the secret outside chat and version control.
|
||||||
|
2. Place it only in the runtime environment or secret manager.
|
||||||
|
3. Sync recoverable copies into an encrypted secrets repository.
|
||||||
|
4. Confirm Git stores ciphertext rather than plaintext.
|
||||||
|
5. Scan staged files and outgoing history before push.
|
||||||
|
6. If exposure occurs, rotate first, then clean current files and rewrite history.
|
||||||
|
7. Re-clone and scan the rewritten remote before considering remediation complete.
|
||||||
|
|
||||||
|
Removing a value from the latest commit does not remove it from Git history.
|
||||||
|
|
||||||
|
## Repository controls
|
||||||
|
|
||||||
|
The public repository uses two checks:
|
||||||
|
|
||||||
|
- `scripts/check-public-safety.py` rejects infrastructure identifiers, private paths, key-like files, and common secret patterns.
|
||||||
|
- Gitleaks scans complete Git history in CI.
|
||||||
|
|
||||||
|
The scanner is deliberately conservative. Placeholder examples should use unmistakable values such as:
|
||||||
|
|
||||||
|
```text
|
||||||
|
CHANGE_ME_DATABASE_PASSWORD
|
||||||
|
CHANGE_ME_API_TOKEN
|
||||||
|
https://service.example.net
|
||||||
|
/home/operator/application
|
||||||
|
/mnt/storage/application-data
|
||||||
|
```
|
||||||
|
|
||||||
|
## Identity and ingress
|
||||||
|
|
||||||
|
- Public application access passes through authenticated ingress.
|
||||||
|
- SSO policy is applied before protected applications.
|
||||||
|
- Administrative interfaces stay on private paths.
|
||||||
|
- Tunnel credentials are isolated from application configuration.
|
||||||
|
- Direct exposure is reserved for protocols that cannot traverse the authenticated HTTP path and is documented separately.
|
||||||
|
|
||||||
|
## Workload isolation
|
||||||
|
|
||||||
|
- Untrusted clients and lab systems cannot initiate general access to production lanes.
|
||||||
|
- Databases are not published to the internet.
|
||||||
|
- Containers receive only required networks and mounts.
|
||||||
|
- Runtime control sockets are avoided or proxied with restricted permissions.
|
||||||
|
- High-risk workloads use separate trust domains rather than relying on process isolation alone.
|
||||||
|
|
||||||
|
## Reporting
|
||||||
|
|
||||||
|
Do not open a public issue containing a suspected secret. Revoke it first and use a private contact channel for disclosure.
|
||||||
168
docs/services.md
Normal file
168
docs/services.md
Normal file
@@ -0,0 +1,168 @@
|
|||||||
|
# Current Service Placement
|
||||||
|
|
||||||
|
This is a public-safe service inventory verified against the live hosts during the September 2026 refresh. Versions, ports, addresses, domains, device IDs, and internal paths are intentionally omitted.
|
||||||
|
|
||||||
|
## Primary application host
|
||||||
|
|
||||||
|
### Core platform
|
||||||
|
|
||||||
|
- TrueNAS SCALE
|
||||||
|
- Docker application runtime
|
||||||
|
- shared PostgreSQL, MariaDB, and Redis
|
||||||
|
- Traefik internal reverse proxy
|
||||||
|
- Newt tunnel connector
|
||||||
|
- Tailscale private-overlay connector
|
||||||
|
- Authentik identity and SSO
|
||||||
|
- Authelia retained for limited legacy migration paths
|
||||||
|
- Headscale and Headplane pilot control plane
|
||||||
|
- Gitea source hosting
|
||||||
|
- Dockhand and Homepage administration interfaces
|
||||||
|
|
||||||
|
### AI and search
|
||||||
|
|
||||||
|
- Ollama always-on model serving
|
||||||
|
- LiteLLM model gateway
|
||||||
|
- Open WebUI
|
||||||
|
- Qdrant vector database
|
||||||
|
- SearXNG metasearch
|
||||||
|
- Firecrawl API, browser worker, queue, and supporting data services
|
||||||
|
- a VPN-isolated search egress path
|
||||||
|
|
||||||
|
### Media and libraries
|
||||||
|
|
||||||
|
- Plex
|
||||||
|
- Tautulli
|
||||||
|
- Audiobookshelf
|
||||||
|
- Calibre Web Automated
|
||||||
|
- Seerr
|
||||||
|
- Immich application and machine-learning services
|
||||||
|
- RomM
|
||||||
|
- GameVault
|
||||||
|
- Dispatcharr
|
||||||
|
|
||||||
|
Large libraries remain on the storage host and are mounted by the application host.
|
||||||
|
|
||||||
|
### Productivity and household applications
|
||||||
|
|
||||||
|
- Paperless-ngx with document conversion and extraction sidecars
|
||||||
|
- Karakeep with browser and search sidecars
|
||||||
|
- n8n
|
||||||
|
- KitchenOwl
|
||||||
|
- Donetick and a dashboard bridge
|
||||||
|
- Shlink and its web client
|
||||||
|
- Qui
|
||||||
|
- Scholarsome
|
||||||
|
- Doris Barbell
|
||||||
|
- Kima Hub
|
||||||
|
|
||||||
|
### Monitoring and notification
|
||||||
|
|
||||||
|
- Grafana
|
||||||
|
- Prometheus
|
||||||
|
- Loki
|
||||||
|
- Promtail
|
||||||
|
- cAdvisor
|
||||||
|
- node exporter
|
||||||
|
- Netdata
|
||||||
|
- Uptime Kuma
|
||||||
|
- Gotify
|
||||||
|
|
||||||
|
### Radio visibility
|
||||||
|
|
||||||
|
- MeshMonitor
|
||||||
|
- local map-tile service
|
||||||
|
|
||||||
|
## Storage and ingestion host
|
||||||
|
|
||||||
|
### Data plane
|
||||||
|
|
||||||
|
- ZFS-backed bulk storage
|
||||||
|
- application-data and media datasets
|
||||||
|
- snapshot and backup targets
|
||||||
|
- synchronized school/document storage
|
||||||
|
|
||||||
|
### Media ingestion
|
||||||
|
|
||||||
|
- Sonarr, including a separate anime workflow
|
||||||
|
- Radarr
|
||||||
|
- Lidarr
|
||||||
|
- Readarr variants
|
||||||
|
- Prowlarr
|
||||||
|
- Bazarr
|
||||||
|
- qBittorrent through a VPN gateway
|
||||||
|
- qbit_manage
|
||||||
|
- Unpackerr
|
||||||
|
- Autobrr
|
||||||
|
- Notifiarr
|
||||||
|
- Shelfmark
|
||||||
|
|
||||||
|
### Supporting services
|
||||||
|
|
||||||
|
- Syncthing hub
|
||||||
|
- RustDesk rendezvous and relay
|
||||||
|
- CPU text reranker
|
||||||
|
- secondary Technitium DNS replica
|
||||||
|
- Netdata
|
||||||
|
- Hawser
|
||||||
|
- Newt connector
|
||||||
|
|
||||||
|
## Offline and resilience host
|
||||||
|
|
||||||
|
### Offline knowledge platform
|
||||||
|
|
||||||
|
- Project N.O.M.A.D. administration layer
|
||||||
|
- Kiwix
|
||||||
|
- two Kolibri generations for retained content compatibility
|
||||||
|
- CyberChef
|
||||||
|
- Flatnotes
|
||||||
|
- MeshCore Web
|
||||||
|
- local Ollama and Qdrant
|
||||||
|
|
||||||
|
Project N.O.M.A.D.-managed containers are treated as an appliance layer and are not manually rebuilt by general maintenance automation.
|
||||||
|
|
||||||
|
### Local agents and household tools
|
||||||
|
|
||||||
|
- Honcho API, deriver, PostgreSQL/Vector database, and Redis
|
||||||
|
- Doris Schoolhouse
|
||||||
|
- Doris Kitchen
|
||||||
|
- CPU Whisper service
|
||||||
|
- LocalSend trusted intake
|
||||||
|
- Hawser
|
||||||
|
- node exporter
|
||||||
|
|
||||||
|
### Resilience and hardware-adjacent services
|
||||||
|
|
||||||
|
- Home Assistant OS in a KVM virtual machine with a dedicated Zigbee radio
|
||||||
|
- secondary Technitium DNS replica
|
||||||
|
- MeshCore companion observer as a system service
|
||||||
|
- Pelican/Wings game-server management
|
||||||
|
- Newt tunnel connector
|
||||||
|
|
||||||
|
## GPU operator workstation
|
||||||
|
|
||||||
|
- CachyOS desktop
|
||||||
|
- high-performance NVIDIA GPU compute
|
||||||
|
- optional heavy Ollama and speech workloads
|
||||||
|
- Docker runtime
|
||||||
|
- printing service
|
||||||
|
- private-overlay access
|
||||||
|
- file-synchronization client
|
||||||
|
- 3D-printing and slicing applications
|
||||||
|
|
||||||
|
The workstation is intentionally opportunistic capacity. Background services must continue when it is asleep or offline.
|
||||||
|
|
||||||
|
## Low-power edge node
|
||||||
|
|
||||||
|
- weather collection and dispatch
|
||||||
|
- mesh-radio/repeater support
|
||||||
|
- wired server-lane placement
|
||||||
|
- low-power always-on operation
|
||||||
|
|
||||||
|
## Placement rules
|
||||||
|
|
||||||
|
1. Shared applications and databases prefer the primary application host.
|
||||||
|
2. Large data and media ingestion stay close to bulk storage.
|
||||||
|
3. Offline knowledge and resilience services remain independent of the primary host.
|
||||||
|
4. Heavy GPU work may use the workstation, but routine automation cannot require it.
|
||||||
|
5. Hardware-adjacent radio and weather services stay on low-power edge nodes.
|
||||||
|
6. DNS, file synchronization, and remote support span hosts to avoid a single failure domain.
|
||||||
6
examples/compose/.env.example
Normal file
6
examples/compose/.env.example
Normal file
@@ -0,0 +1,6 @@
|
|||||||
|
# Copy this file to .env and replace the placeholders locally.
|
||||||
|
# Never commit the resulting .env file.
|
||||||
|
DATABASE_NAME=showcase
|
||||||
|
DATABASE_USER=showcase
|
||||||
|
DATABASE_PASSWORD=CHANGE_ME_DATABASE_PASSWORD
|
||||||
|
INGRESS_NETWORK=example-ingress
|
||||||
41
examples/compose/docker-compose.yml
Normal file
41
examples/compose/docker-compose.yml
Normal file
@@ -0,0 +1,41 @@
|
|||||||
|
services:
|
||||||
|
database:
|
||||||
|
image: postgres:17-alpine
|
||||||
|
restart: unless-stopped
|
||||||
|
environment:
|
||||||
|
POSTGRES_DB: ${DATABASE_NAME}
|
||||||
|
POSTGRES_USER: ${DATABASE_USER}
|
||||||
|
POSTGRES_PASSWORD: ${DATABASE_PASSWORD}
|
||||||
|
volumes:
|
||||||
|
- database-data:/var/lib/postgresql/data
|
||||||
|
networks:
|
||||||
|
- data
|
||||||
|
healthcheck:
|
||||||
|
test: ["CMD-SHELL", "pg_isready -U $${POSTGRES_USER} -d $${POSTGRES_DB}"]
|
||||||
|
interval: 10s
|
||||||
|
timeout: 5s
|
||||||
|
retries: 5
|
||||||
|
|
||||||
|
application:
|
||||||
|
image: traefik/whoami:v1.11
|
||||||
|
restart: unless-stopped
|
||||||
|
depends_on:
|
||||||
|
database:
|
||||||
|
condition: service_healthy
|
||||||
|
networks:
|
||||||
|
- data
|
||||||
|
- ingress
|
||||||
|
expose:
|
||||||
|
- "80"
|
||||||
|
labels:
|
||||||
|
showcase.description: "Example internal application"
|
||||||
|
|
||||||
|
networks:
|
||||||
|
data:
|
||||||
|
internal: true
|
||||||
|
ingress:
|
||||||
|
name: ${INGRESS_NETWORK}
|
||||||
|
external: true
|
||||||
|
|
||||||
|
volumes:
|
||||||
|
database-data:
|
||||||
162
scripts/check-public-safety.py
Executable file
162
scripts/check-public-safety.py
Executable file
@@ -0,0 +1,162 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""Fail when tracked content is unsafe for a public showcase."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import math
|
||||||
|
import re
|
||||||
|
import subprocess
|
||||||
|
import sys
|
||||||
|
from collections import Counter
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
ROOT = Path(__file__).resolve().parents[1]
|
||||||
|
SELF = "scripts/check-public-safety.py"
|
||||||
|
|
||||||
|
PLACEHOLDER_WORDS = (
|
||||||
|
"change_me",
|
||||||
|
"redacted",
|
||||||
|
"placeholder",
|
||||||
|
"example",
|
||||||
|
"your_token",
|
||||||
|
"your_key",
|
||||||
|
"***",
|
||||||
|
)
|
||||||
|
|
||||||
|
ANY_IPV4 = re.compile(r"(?<![\d.])(?:\d{1,3}\.){3}\d{1,3}(?![\d.])")
|
||||||
|
ANY_IPV6 = re.compile(r"(?i)(?<![0-9a-f:])(?:[0-9a-f]{0,4}:){2,8}[0-9a-f]{0,4}(?![0-9a-f:])")
|
||||||
|
MAC = re.compile(r"(?i)\b(?:[0-9a-f]{2}[:-]){5}[0-9a-f]{2}\b")
|
||||||
|
UUID = re.compile(r"(?i)\b[0-9a-f]{8}-[0-9a-f]{4}-[1-5][0-9a-f]{3}-[89ab][0-9a-f]{3}-[0-9a-f]{12}\b")
|
||||||
|
INTERNAL_DOMAIN = re.compile(r"(?i)\b[a-z0-9.-]+\.(?:local|lan|internal|home|private)\b")
|
||||||
|
PRIVATE_HOME = re.compile(r"(?i)/home/(?!user\b|example\b|operator\b)[a-z0-9._-]+\b")
|
||||||
|
PRIVATE_MOUNT = re.compile(r"(?i)/mnt/(?!storage\b|media\b|backups\b|example\b)[a-z0-9._-]+\b")
|
||||||
|
PRIVATE_OPT = re.compile(r"(?i)/opt/(?!example\b|application\b)[a-z0-9._-]+\b")
|
||||||
|
PRIVATE_KEY = re.compile(r"-----BEGIN [A-Z0-9 ]*PRIVATE KEY-----", re.I)
|
||||||
|
EMAIL = re.compile(r"(?i)\b[A-Z0-9._%+-]+@([A-Z0-9.-]+\.[A-Z]{2,})\b")
|
||||||
|
SECRET_ASSIGNMENT = re.compile(
|
||||||
|
r"(?i)\b(?:api[_-]?(?:key|token)|access[_-]?token|auth[_-]?token|token|password|passwd|"
|
||||||
|
r"client[_-]?secret|cookie|private[_-]?key)\b\s*[:=]\s*[\"']?([^\s\"',}]+)"
|
||||||
|
)
|
||||||
|
AUTHORIZATION = re.compile(
|
||||||
|
r"(?i)authorization\s*[:=]\s*(?:bearer|token|basic)\s+([^\s\"']+)"
|
||||||
|
)
|
||||||
|
OPAQUE = re.compile(r"\b[A-Za-z0-9_+/=-]{32,}\b")
|
||||||
|
|
||||||
|
RISKY_SUFFIXES = {
|
||||||
|
".pem", ".key", ".p12", ".pfx", ".kdbx", ".ovpn", ".mobileconfig"
|
||||||
|
}
|
||||||
|
RISKY_NAMES = {
|
||||||
|
".env", "id_rsa", "id_ed25519", "id_ecdsa", "id_dsa", "wg0.conf"
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def tracked_files() -> list[Path]:
|
||||||
|
result = subprocess.run(
|
||||||
|
["git", "-C", str(ROOT), "ls-files", "-z"],
|
||||||
|
check=True,
|
||||||
|
capture_output=True,
|
||||||
|
)
|
||||||
|
return [ROOT / value.decode() for value in result.stdout.split(b"\0") if value]
|
||||||
|
|
||||||
|
|
||||||
|
def is_placeholder(value: str) -> bool:
|
||||||
|
lowered = value.lower()
|
||||||
|
return value.startswith("${") or any(word in lowered for word in PLACEHOLDER_WORDS)
|
||||||
|
|
||||||
|
|
||||||
|
def entropy(value: str) -> float:
|
||||||
|
counts = Counter(value)
|
||||||
|
total = len(value)
|
||||||
|
return -sum((count / total) * math.log2(count / total) for count in counts.values())
|
||||||
|
|
||||||
|
|
||||||
|
def main() -> int:
|
||||||
|
findings: set[str] = set()
|
||||||
|
files = tracked_files()
|
||||||
|
|
||||||
|
for path in files:
|
||||||
|
relative = path.relative_to(ROOT).as_posix()
|
||||||
|
if relative == SELF:
|
||||||
|
continue
|
||||||
|
|
||||||
|
name = path.name.lower()
|
||||||
|
if name.startswith(".env") and name != ".env.example":
|
||||||
|
findings.add(f"{relative}: prohibited environment filename")
|
||||||
|
if name in RISKY_NAMES or re.fullmatch(r"id_[a-z0-9_-]+", name):
|
||||||
|
findings.add(f"{relative}: prohibited credential filename")
|
||||||
|
if path.suffix.lower() in RISKY_SUFFIXES:
|
||||||
|
findings.add(f"{relative}: prohibited key/config suffix")
|
||||||
|
if re.fullmatch(r"wg\d+\.conf", name):
|
||||||
|
findings.add(f"{relative}: prohibited VPN configuration filename")
|
||||||
|
|
||||||
|
try:
|
||||||
|
data = path.read_bytes()
|
||||||
|
except OSError as exc:
|
||||||
|
findings.add(f"{relative}: unreadable: {exc}")
|
||||||
|
continue
|
||||||
|
if b"\0" in data[:4096]:
|
||||||
|
continue
|
||||||
|
|
||||||
|
for line_number, line in enumerate(data.decode("utf-8", errors="ignore").splitlines(), 1):
|
||||||
|
prefix = f"{relative}:{line_number}"
|
||||||
|
|
||||||
|
if ANY_IPV4.search(line):
|
||||||
|
findings.add(f"{prefix}: literal IPv4 address")
|
||||||
|
if ANY_IPV6.search(line):
|
||||||
|
findings.add(f"{prefix}: literal IPv6 address")
|
||||||
|
if MAC.search(line):
|
||||||
|
findings.add(f"{prefix}: MAC address")
|
||||||
|
if UUID.search(line):
|
||||||
|
findings.add(f"{prefix}: UUID/device identifier")
|
||||||
|
if INTERNAL_DOMAIN.search(line):
|
||||||
|
findings.add(f"{prefix}: internal domain")
|
||||||
|
if PRIVATE_HOME.search(line):
|
||||||
|
findings.add(f"{prefix}: private home path")
|
||||||
|
if PRIVATE_MOUNT.search(line):
|
||||||
|
findings.add(f"{prefix}: private mount path")
|
||||||
|
if PRIVATE_OPT.search(line):
|
||||||
|
findings.add(f"{prefix}: private application path")
|
||||||
|
if PRIVATE_KEY.search(line):
|
||||||
|
findings.add(f"{prefix}: private-key block")
|
||||||
|
|
||||||
|
for match in EMAIL.finditer(line):
|
||||||
|
domain = match.group(1).lower()
|
||||||
|
if not domain.endswith(("example.com", "example.net", "example.org")):
|
||||||
|
findings.add(f"{prefix}: non-example email address")
|
||||||
|
|
||||||
|
for match in SECRET_ASSIGNMENT.finditer(line):
|
||||||
|
value = match.group(1)
|
||||||
|
if not is_placeholder(value):
|
||||||
|
findings.add(f"{prefix}: non-placeholder secret assignment")
|
||||||
|
|
||||||
|
for match in AUTHORIZATION.finditer(line):
|
||||||
|
value = match.group(1)
|
||||||
|
if not is_placeholder(value):
|
||||||
|
findings.add(f"{prefix}: authorization material")
|
||||||
|
|
||||||
|
for candidate in OPAQUE.findall(line):
|
||||||
|
if is_placeholder(candidate):
|
||||||
|
continue
|
||||||
|
# Commit hashes and content digests are still identifiers; require a label.
|
||||||
|
labelled_hash = re.search(
|
||||||
|
r"(?i)\b(?:sha(?:1|256|512)|digest|commit|checksum|example[_-]?hash)\b",
|
||||||
|
line,
|
||||||
|
)
|
||||||
|
if labelled_hash and re.fullmatch(r"[0-9a-fA-F]{32,128}", candidate):
|
||||||
|
continue
|
||||||
|
if entropy(candidate) >= 3.5:
|
||||||
|
findings.add(f"{prefix}: opaque high-entropy value")
|
||||||
|
break
|
||||||
|
|
||||||
|
if findings:
|
||||||
|
print("Public-safety scan failed:")
|
||||||
|
for finding in sorted(findings):
|
||||||
|
print(f"- {finding}")
|
||||||
|
return 1
|
||||||
|
|
||||||
|
print(f"Public-safety scan passed: {len(files)} tracked files checked")
|
||||||
|
return 0
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
sys.exit(main())
|
||||||
Reference in New Issue
Block a user