Publish sanitized current homelab architecture
Some checks failed
public-safety / privacy-and-secret-scan (push) Has been cancelled

This commit is contained in:
fizzlepoof
2026-09-08 21:47:41 +00:00
commit cc35523888
13 changed files with 901 additions and 0 deletions

24
.github/workflows/public-safety.yml vendored Normal file
View File

@@ -0,0 +1,24 @@
name: public-safety
on:
push:
pull_request:
permissions:
contents: read
jobs:
privacy-and-secret-scan:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Check public-safe content
run: python3 scripts/check-public-safety.py
- name: Scan Git history for secrets
uses: gitleaks/gitleaks-action@v2
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

19
.gitignore vendored Normal file
View File

@@ -0,0 +1,19 @@
# Local runtime files
.env
.env.*
!.env.example
# Python
__pycache__/
*.py[cod]
.venv/
# Editors and operating systems
.idea/
.vscode/
.DS_Store
# Generated scans and local artifacts
*.sarif
*.log
reports/

21
LICENSE Normal file
View File

@@ -0,0 +1,21 @@
MIT License
Copyright (c) 2026 fizzlepoof
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

88
README.md Normal file
View File

@@ -0,0 +1,88 @@
# Homelab Showcase
A public-safe overview of a real multi-host homelab built around segmented networking, self-hosted applications, resilient storage, local AI, offline services, and low-power edge nodes.
> This repository is intentionally **not** the operational source of truth. Exact addresses, domains, account names, device identifiers, locations, mount paths, credentials, and raw controller exports are omitted or replaced with examples.
## What this lab is designed to do
- Keep household services useful during an internet outage
- Separate trusted devices, servers, IoT, cameras, guests, and lab workloads
- Centralize applications without making one server responsible for every failure domain
- Keep bulk storage close to media ingestion and backup workflows
- Route local AI work to the most appropriate always-on or high-performance host
- Publish selected services through authenticated tunnels instead of broad inbound exposure
- Preserve recovery options with encrypted secrets, snapshots, versioned sync, and documented rollback steps
## Current architecture
| Node | Platform | Primary role |
|---|---|---|
| **Application host** | TrueNAS SCALE | Primary application, database, identity, observability, media, and shared AI host |
| **Storage host** | Storage-focused Linux server | Bulk storage, media-ingestion pipeline, synchronization hub, remote-support relay, and selected CPU services |
| **Resilience host** | Ubuntu Server | Offline knowledge, resilient local tools, home automation VM, game orchestration, and secondary AI/DNS services |
| **GPU workstation** | CachyOS workstation | Operator workstation and opportunistic high-performance GPU compute |
| **Edge node** | Low-power Linux node | Weather dispatch and mesh-radio edge services |
See [Architecture](docs/architecture.md) for the data flow and [Services](docs/services.md) for the current service placement.
## Design highlights
- **Segmented UniFi network:** management, trusted clients, servers, IoT, cameras, guests, and quarantine are separate policy lanes.
- **Authenticated ingress:** selected applications use a tunnel gateway, an internal reverse proxy, and SSO. Most services have no direct internet-facing listener.
- **Three-node DNS design:** one authoritative configuration source synchronizes to two backup resolvers; public fallback DNS is treated separately from private-zone continuity.
- **Shared databases:** PostgreSQL, MariaDB, and Redis provide a managed data layer for compatible application stacks.
- **Split storage and compute:** application compute is concentrated on the primary host while large media/photo datasets stay on the storage server.
- **Tiered local AI:** an always-on light tier handles routine work; the GPU workstation is optional capacity rather than a hard dependency.
- **Offline-first node:** documentation, educational content, notes, utilities, and selected models remain available locally.
- **Encrypted operations:** runtime secrets live outside application repositories and are encrypted before entering version control.
## Repository map
```text
docs/
architecture.md High-level topology and request flows
networking.md Segmentation and access-control model
operations.md Deployment, verification, backup, and rollback patterns
security.md Public/private boundary and secret-handling rules
services.md Current service placement by host
examples/
compose/ Sanitized Compose and environment examples
scripts/
check-public-safety.py
```
## Reusable patterns
- [Sanitized Compose example](examples/compose/docker-compose.yml)
- [Environment template](examples/compose/.env.example)
- [Public-safety scanner](scripts/check-public-safety.py)
- [Operational practices](docs/operations.md)
- [Security model](docs/security.md)
## Public-safety policy
Every commit is checked for:
- private or management addresses
- MAC addresses and device identifiers
- internal domains and personal filesystem paths
- private-key material, tokens, passwords, and authorization headers
- real `.env` files and key-bearing file types
Gitleaks also scans the complete Git history in CI. See [Security](docs/security.md).
## Scope
This repository demonstrates architecture and operating patterns. It deliberately excludes:
- deployable production secrets
- exact network coordinates
- raw exports, backups, logs, and incident artifacts
- household or personal records
- device serial numbers and persistent client identifiers
- live tunnel, VPN, DNS, or identity-provider configuration
## License
MIT — see [LICENSE](LICENSE).

17
SECURITY.md Normal file
View File

@@ -0,0 +1,17 @@
# Security Policy
## Scope
This repository contains public-safe architecture documentation and sanitized examples only. It does not accept production credentials, raw infrastructure exports, private topology, or personal data.
## Reporting a problem
If you find sensitive information in this repository:
1. Do not quote it in a public issue.
2. Contact the repository owner through an already established private channel.
3. Include only the affected path and the type of exposure until the value has been revoked.
A confirmed credential exposure is handled by rotating the credential first, removing it from current files, rewriting affected Git history, and verifying a fresh clone.
For the full repository policy, see [docs/security.md](docs/security.md).

141
docs/architecture.md Normal file
View File

@@ -0,0 +1,141 @@
# Architecture
## Topology
```mermaid
flowchart TB
Internet((Internet))
Edge[Authenticated tunnel gateway]
SSO[Identity and SSO]
Proxy[Internal reverse proxy]
subgraph Home[Segmented home network]
subgraph Primary[Primary application host]
Apps[Household and productivity apps]
Data[Shared databases and caches]
Media[Media, photos, and documents]
Observe[Monitoring and dashboards]
AI[Always-on AI and search]
end
subgraph Storage[Storage and ingestion host]
Pool[Bulk ZFS storage]
Arr[Media automation and download pipeline]
Sync[File synchronization and backups]
Remote[Remote-support relay]
end
subgraph Resilience[Offline and resilience host]
Offline[Offline knowledge and education]
HomeAuto[Home automation VM]
Games[Game-server control plane]
Secondary[Backup DNS and local AI]
Agents[Automation and agent services]
end
Workstation[GPU operator workstation]
EdgeNode[Low-power weather and radio node]
end
Internet --> Edge --> Proxy
Proxy --> SSO
SSO --> Apps
Apps --> Data
Apps --> Pool
Media --> Pool
Arr --> Pool
Observe --> Apps
AI -. optional heavy jobs .-> Workstation
Secondary -. resilient local path .-> AI
Agents --> Apps
Sync --> Workstation
EdgeNode --> Agents
Remote -. private overlay .-> Workstation
```
## Host responsibilities
### Primary application host
The primary host carries the application plane:
- shared PostgreSQL, MariaDB, and Redis
- identity, SSO, ingress routing, and tunnel clients
- media front ends, photo management, and document management
- dashboards, monitoring, metrics, logs, and notifications
- household applications and workflow automation
- light-tier model serving, search, RAG, and vector storage
- self-hosted development and administration tools
The host is intentionally not the only copy of storage, DNS, or offline knowledge.
### Storage and ingestion host
The storage host owns the large-capacity data plane and workloads that benefit from data locality:
- ZFS-backed bulk datasets
- media acquisition, organization, and post-processing
- synchronized document storage with versioning
- backup targets and retained application data
- private remote-support rendezvous and relay
- selected CPU-only inference utilities
- a secondary DNS replica
### Offline and resilience host
The resilience host is designed to remain useful when public services are unavailable:
- offline reference libraries and educational content
- local notes and browser utilities
- local model and vector services
- agent memory and household helper applications
- Home Assistant running in a dedicated virtual machine
- game-server management
- mesh-radio observation
- a secondary DNS replica
### GPU workstation
The workstation provides high-performance GPU capacity for interactive or heavy jobs. Routine background tasks must continue when it is asleep or offline.
### Low-power edge node
The edge node handles always-on weather and mesh-radio functions close to the attached hardware. It reports upstream but is not part of the main application failure domain.
## Request flows
### External application request
```text
client → tunnel gateway → internal reverse proxy → SSO policy → application
```
The tunnel endpoint is the only public routing layer. Application containers do not require direct inbound router exposure.
### Internal application request
```text
trusted client → internal DNS → application or reverse proxy → service
```
### AI request
```text
application → model gateway → always-on local model
↘ optional GPU workstation
```
### Storage request
```text
application host → authenticated network storage mount → storage dataset
```
## Failure-domain rules
1. Losing the GPU workstation must not stop routine automation.
2. Losing public DNS must not be confused with losing the private DNS zone.
3. Losing the primary application host must not destroy bulk datasets or backups.
4. A tunnel or SSO failure must not expose an application directly.
5. The offline host must retain useful local content without WAN access.
6. Changes to storage, networking, identity, and databases require explicit rollback points.

69
docs/networking.md Normal file
View File

@@ -0,0 +1,69 @@
# Networking
## Segmentation model
The network uses purpose-specific policy lanes rather than one trusted broadcast domain.
| Lane | Intended occupants | Default posture |
|---|---|---|
| **Management** | Gateway, switches, access points, controllers | Administrative access only |
| **Trusted** | Human-operated laptops, phones, and workstations | May initiate approved internal connections |
| **Servers** | Application, storage, DNS, and automation hosts | Explicit service exposure |
| **IoT** | Appliances and embedded clients | Restricted from internal networks by default |
| **Cameras** | Video and security devices | More restrictive than general IoT |
| **Guest** | Visitors and temporary clients | Internet only |
| **Quarantine** | Legacy or unidentified devices | Minimal DNS, time, and internet access |
| **Lab** | Disposable security and development systems | Default deny toward production |
Exact VLAN identifiers, addressing, SSIDs, and device assignments are intentionally absent from this public repository.
## Policy principles
- Inter-lane access is denied unless a specific workflow requires it.
- Management interfaces are never reachable from guest, IoT, camera, or lab lanes.
- IoT exceptions are limited to named services and destinations.
- Discovery protocols are bridged only when a real use case requires them.
- DNS, DHCP, and time synchronization are treated as explicit dependencies.
- A trusted operator path stays available during every network change.
- Client migrations happen in small batches with rollback after each batch.
## DNS resilience
The lab uses three synchronized internal resolvers:
1. a primary configuration source on the application host
2. one replica on the offline/resilience host
3. one replica on the storage host
The replicas receive configuration on a schedule and are verified from another host. External recursive resolvers may provide public-name fallback, but they cannot replace authoritative answers for the private zone.
## Remote access
Two complementary mechanisms are used:
- **Authenticated application ingress:** tunnel gateway → reverse proxy → SSO → selected service
- **Private administrative overlay:** device-to-device access for operators and private support workflows
Most applications do not expose router-forwarded ports. Protocols that cannot use the HTTP tunnel are handled as narrow, documented exceptions.
## Container networking
Compose projects use named external networks for shared dependencies:
- a data network for databases and caches
- an ingress network for explicitly published applications
- optional service-family networks for AI and observability
Applications join only the networks they need. The reverse proxy does not receive unrestricted control of the container runtime.
## Validation gates
After a network change, verify:
1. gateway and controller access from a trusted client
2. DHCP lease and intended lane placement
3. private and public DNS answers
4. the exact application protocol, not only ping
5. cross-lane allow and deny behavior
6. recovery after client reassociation or host reboot
7. absence of broad temporary allow rules

78
docs/operations.md Normal file
View File

@@ -0,0 +1,78 @@
# Operations
## Deployment order
A predictable dependency order reduces partial deployments:
1. storage mounts and host prerequisites
2. shared databases and caches
3. ingress, identity, DNS, and management services
4. observability and notifications
5. application stacks
6. optional AI, media, and edge integrations
## Change workflow
1. **Read first.** Capture current host, service, network, and storage state.
2. **Define scope.** Name the exact stack, files, and expected result.
3. **Create rollback.** Back up configuration and state before mutation.
4. **Validate configuration.** Render Compose, parse structured files, and run syntax checks.
5. **Apply narrowly.** Restart or recreate only the service that consumes the change.
6. **Verify from the consumer side.** A process being `running` is not proof that the workflow works.
7. **Read back external state.** Confirm the controller, API, repository, or database actually retained the change.
8. **Document the result.** Record current truth, rollback location, and deferred work.
## Compose validation
A sanitized example stack is provided under `examples/compose/`.
```bash
cp examples/compose/.env.example examples/compose/.env
# Replace placeholders only in the untracked .env file.
docker compose --env-file examples/compose/.env \
-f examples/compose/docker-compose.yml config
```
Never commit the generated `.env` file.
## Service verification
Use multiple layers of evidence:
- container or service state
- health endpoint
- recent logs
- dependency connectivity
- application-level request
- persistence after restart
- monitoring visibility
For stateful applications, also check a representative database object or file after restart.
## Backup model
- filesystem snapshots protect bulk datasets
- application-aware exports protect databases and stateful services
- encrypted Git stores recoverable configuration secrets
- normal Git stores source, sanitized templates, and operator documentation
- synchronized user files use versioning in addition to storage snapshots
- restore tests are scheduled; backup creation alone is not treated as recovery proof
## Storage and data locality
Large data stays on the storage host. Applications consume it over authenticated mounts where practical. Ingestion workloads remain close to the data when moving them would create avoidable network traffic or fragile cross-host dependencies.
## Monitoring and notifications
Metrics, logs, uptime checks, and push notifications are separate components so one dashboard failure does not erase the underlying evidence. Alerts should be low-volume, actionable, and routed to an operator channel with enough context to diagnose.
## Updating documentation
Documentation is divided into:
- **current state:** what is running and where
- **operating procedure:** how to change and verify it
- **historical evidence:** dated incident or migration records
- **future plan:** proposed work that is not yet live
Dated historical notes should never read like the active backlog. Public documentation describes roles and patterns; exact operational coordinates remain private.

67
docs/security.md Normal file
View File

@@ -0,0 +1,67 @@
# Security
## Public/private boundary
This public repository contains architectural patterns, sanitized examples, and high-level service placement. It is not used for production deployment.
The private operational repository contains exact topology, runtime paths, deployment notes, and rollback artifacts. A separate encrypted repository stores environment files and other secrets.
## Never publish
- credentials, tokens, cookies, private keys, recovery codes, or session material
- real `.env` files
- internal or management addresses
- MAC addresses, serial numbers, client IDs, or tunnel IDs
- real domains, SSIDs, account names, or personal filesystem paths
- physical locations or household records
- raw controller exports, logs, database dumps, or backups
- screenshots containing dashboards, account menus, or browser sessions
## Secret workflow
1. Generate or receive the secret outside chat and version control.
2. Place it only in the runtime environment or secret manager.
3. Sync recoverable copies into an encrypted secrets repository.
4. Confirm Git stores ciphertext rather than plaintext.
5. Scan staged files and outgoing history before push.
6. If exposure occurs, rotate first, then clean current files and rewrite history.
7. Re-clone and scan the rewritten remote before considering remediation complete.
Removing a value from the latest commit does not remove it from Git history.
## Repository controls
The public repository uses two checks:
- `scripts/check-public-safety.py` rejects infrastructure identifiers, private paths, key-like files, and common secret patterns.
- Gitleaks scans complete Git history in CI.
The scanner is deliberately conservative. Placeholder examples should use unmistakable values such as:
```text
CHANGE_ME_DATABASE_PASSWORD
CHANGE_ME_API_TOKEN
https://service.example.net
/home/operator/application
/mnt/storage/application-data
```
## Identity and ingress
- Public application access passes through authenticated ingress.
- SSO policy is applied before protected applications.
- Administrative interfaces stay on private paths.
- Tunnel credentials are isolated from application configuration.
- Direct exposure is reserved for protocols that cannot traverse the authenticated HTTP path and is documented separately.
## Workload isolation
- Untrusted clients and lab systems cannot initiate general access to production lanes.
- Databases are not published to the internet.
- Containers receive only required networks and mounts.
- Runtime control sockets are avoided or proxied with restricted permissions.
- High-risk workloads use separate trust domains rather than relying on process isolation alone.
## Reporting
Do not open a public issue containing a suspected secret. Revoke it first and use a private contact channel for disclosure.

168
docs/services.md Normal file
View File

@@ -0,0 +1,168 @@
# Current Service Placement
This is a public-safe service inventory verified against the live hosts during the September 2026 refresh. Versions, ports, addresses, domains, device IDs, and internal paths are intentionally omitted.
## Primary application host
### Core platform
- TrueNAS SCALE
- Docker application runtime
- shared PostgreSQL, MariaDB, and Redis
- Traefik internal reverse proxy
- Newt tunnel connector
- Tailscale private-overlay connector
- Authentik identity and SSO
- Authelia retained for limited legacy migration paths
- Headscale and Headplane pilot control plane
- Gitea source hosting
- Dockhand and Homepage administration interfaces
### AI and search
- Ollama always-on model serving
- LiteLLM model gateway
- Open WebUI
- Qdrant vector database
- SearXNG metasearch
- Firecrawl API, browser worker, queue, and supporting data services
- a VPN-isolated search egress path
### Media and libraries
- Plex
- Tautulli
- Audiobookshelf
- Calibre Web Automated
- Seerr
- Immich application and machine-learning services
- RomM
- GameVault
- Dispatcharr
Large libraries remain on the storage host and are mounted by the application host.
### Productivity and household applications
- Paperless-ngx with document conversion and extraction sidecars
- Karakeep with browser and search sidecars
- n8n
- KitchenOwl
- Donetick and a dashboard bridge
- Shlink and its web client
- Qui
- Scholarsome
- Doris Barbell
- Kima Hub
### Monitoring and notification
- Grafana
- Prometheus
- Loki
- Promtail
- cAdvisor
- node exporter
- Netdata
- Uptime Kuma
- Gotify
### Radio visibility
- MeshMonitor
- local map-tile service
## Storage and ingestion host
### Data plane
- ZFS-backed bulk storage
- application-data and media datasets
- snapshot and backup targets
- synchronized school/document storage
### Media ingestion
- Sonarr, including a separate anime workflow
- Radarr
- Lidarr
- Readarr variants
- Prowlarr
- Bazarr
- qBittorrent through a VPN gateway
- qbit_manage
- Unpackerr
- Autobrr
- Notifiarr
- Shelfmark
### Supporting services
- Syncthing hub
- RustDesk rendezvous and relay
- CPU text reranker
- secondary Technitium DNS replica
- Netdata
- Hawser
- Newt connector
## Offline and resilience host
### Offline knowledge platform
- Project N.O.M.A.D. administration layer
- Kiwix
- two Kolibri generations for retained content compatibility
- CyberChef
- Flatnotes
- MeshCore Web
- local Ollama and Qdrant
Project N.O.M.A.D.-managed containers are treated as an appliance layer and are not manually rebuilt by general maintenance automation.
### Local agents and household tools
- Honcho API, deriver, PostgreSQL/Vector database, and Redis
- Doris Schoolhouse
- Doris Kitchen
- CPU Whisper service
- LocalSend trusted intake
- Hawser
- node exporter
### Resilience and hardware-adjacent services
- Home Assistant OS in a KVM virtual machine with a dedicated Zigbee radio
- secondary Technitium DNS replica
- MeshCore companion observer as a system service
- Pelican/Wings game-server management
- Newt tunnel connector
## GPU operator workstation
- CachyOS desktop
- high-performance NVIDIA GPU compute
- optional heavy Ollama and speech workloads
- Docker runtime
- printing service
- private-overlay access
- file-synchronization client
- 3D-printing and slicing applications
The workstation is intentionally opportunistic capacity. Background services must continue when it is asleep or offline.
## Low-power edge node
- weather collection and dispatch
- mesh-radio/repeater support
- wired server-lane placement
- low-power always-on operation
## Placement rules
1. Shared applications and databases prefer the primary application host.
2. Large data and media ingestion stay close to bulk storage.
3. Offline knowledge and resilience services remain independent of the primary host.
4. Heavy GPU work may use the workstation, but routine automation cannot require it.
5. Hardware-adjacent radio and weather services stay on low-power edge nodes.
6. DNS, file synchronization, and remote support span hosts to avoid a single failure domain.

View File

@@ -0,0 +1,6 @@
# Copy this file to .env and replace the placeholders locally.
# Never commit the resulting .env file.
DATABASE_NAME=showcase
DATABASE_USER=showcase
DATABASE_PASSWORD=CHANGE_ME_DATABASE_PASSWORD
INGRESS_NETWORK=example-ingress

View File

@@ -0,0 +1,41 @@
services:
database:
image: postgres:17-alpine
restart: unless-stopped
environment:
POSTGRES_DB: ${DATABASE_NAME}
POSTGRES_USER: ${DATABASE_USER}
POSTGRES_PASSWORD: ${DATABASE_PASSWORD}
volumes:
- database-data:/var/lib/postgresql/data
networks:
- data
healthcheck:
test: ["CMD-SHELL", "pg_isready -U $${POSTGRES_USER} -d $${POSTGRES_DB}"]
interval: 10s
timeout: 5s
retries: 5
application:
image: traefik/whoami:v1.11
restart: unless-stopped
depends_on:
database:
condition: service_healthy
networks:
- data
- ingress
expose:
- "80"
labels:
showcase.description: "Example internal application"
networks:
data:
internal: true
ingress:
name: ${INGRESS_NETWORK}
external: true
volumes:
database-data:

162
scripts/check-public-safety.py Executable file
View File

@@ -0,0 +1,162 @@
#!/usr/bin/env python3
"""Fail when tracked content is unsafe for a public showcase."""
from __future__ import annotations
import math
import re
import subprocess
import sys
from collections import Counter
from pathlib import Path
ROOT = Path(__file__).resolve().parents[1]
SELF = "scripts/check-public-safety.py"
PLACEHOLDER_WORDS = (
"change_me",
"redacted",
"placeholder",
"example",
"your_token",
"your_key",
"***",
)
ANY_IPV4 = re.compile(r"(?<![\d.])(?:\d{1,3}\.){3}\d{1,3}(?![\d.])")
ANY_IPV6 = re.compile(r"(?i)(?<![0-9a-f:])(?:[0-9a-f]{0,4}:){2,8}[0-9a-f]{0,4}(?![0-9a-f:])")
MAC = re.compile(r"(?i)\b(?:[0-9a-f]{2}[:-]){5}[0-9a-f]{2}\b")
UUID = re.compile(r"(?i)\b[0-9a-f]{8}-[0-9a-f]{4}-[1-5][0-9a-f]{3}-[89ab][0-9a-f]{3}-[0-9a-f]{12}\b")
INTERNAL_DOMAIN = re.compile(r"(?i)\b[a-z0-9.-]+\.(?:local|lan|internal|home|private)\b")
PRIVATE_HOME = re.compile(r"(?i)/home/(?!user\b|example\b|operator\b)[a-z0-9._-]+\b")
PRIVATE_MOUNT = re.compile(r"(?i)/mnt/(?!storage\b|media\b|backups\b|example\b)[a-z0-9._-]+\b")
PRIVATE_OPT = re.compile(r"(?i)/opt/(?!example\b|application\b)[a-z0-9._-]+\b")
PRIVATE_KEY = re.compile(r"-----BEGIN [A-Z0-9 ]*PRIVATE KEY-----", re.I)
EMAIL = re.compile(r"(?i)\b[A-Z0-9._%+-]+@([A-Z0-9.-]+\.[A-Z]{2,})\b")
SECRET_ASSIGNMENT = re.compile(
r"(?i)\b(?:api[_-]?(?:key|token)|access[_-]?token|auth[_-]?token|token|password|passwd|"
r"client[_-]?secret|cookie|private[_-]?key)\b\s*[:=]\s*[\"']?([^\s\"',}]+)"
)
AUTHORIZATION = re.compile(
r"(?i)authorization\s*[:=]\s*(?:bearer|token|basic)\s+([^\s\"']+)"
)
OPAQUE = re.compile(r"\b[A-Za-z0-9_+/=-]{32,}\b")
RISKY_SUFFIXES = {
".pem", ".key", ".p12", ".pfx", ".kdbx", ".ovpn", ".mobileconfig"
}
RISKY_NAMES = {
".env", "id_rsa", "id_ed25519", "id_ecdsa", "id_dsa", "wg0.conf"
}
def tracked_files() -> list[Path]:
result = subprocess.run(
["git", "-C", str(ROOT), "ls-files", "-z"],
check=True,
capture_output=True,
)
return [ROOT / value.decode() for value in result.stdout.split(b"\0") if value]
def is_placeholder(value: str) -> bool:
lowered = value.lower()
return value.startswith("${") or any(word in lowered for word in PLACEHOLDER_WORDS)
def entropy(value: str) -> float:
counts = Counter(value)
total = len(value)
return -sum((count / total) * math.log2(count / total) for count in counts.values())
def main() -> int:
findings: set[str] = set()
files = tracked_files()
for path in files:
relative = path.relative_to(ROOT).as_posix()
if relative == SELF:
continue
name = path.name.lower()
if name.startswith(".env") and name != ".env.example":
findings.add(f"{relative}: prohibited environment filename")
if name in RISKY_NAMES or re.fullmatch(r"id_[a-z0-9_-]+", name):
findings.add(f"{relative}: prohibited credential filename")
if path.suffix.lower() in RISKY_SUFFIXES:
findings.add(f"{relative}: prohibited key/config suffix")
if re.fullmatch(r"wg\d+\.conf", name):
findings.add(f"{relative}: prohibited VPN configuration filename")
try:
data = path.read_bytes()
except OSError as exc:
findings.add(f"{relative}: unreadable: {exc}")
continue
if b"\0" in data[:4096]:
continue
for line_number, line in enumerate(data.decode("utf-8", errors="ignore").splitlines(), 1):
prefix = f"{relative}:{line_number}"
if ANY_IPV4.search(line):
findings.add(f"{prefix}: literal IPv4 address")
if ANY_IPV6.search(line):
findings.add(f"{prefix}: literal IPv6 address")
if MAC.search(line):
findings.add(f"{prefix}: MAC address")
if UUID.search(line):
findings.add(f"{prefix}: UUID/device identifier")
if INTERNAL_DOMAIN.search(line):
findings.add(f"{prefix}: internal domain")
if PRIVATE_HOME.search(line):
findings.add(f"{prefix}: private home path")
if PRIVATE_MOUNT.search(line):
findings.add(f"{prefix}: private mount path")
if PRIVATE_OPT.search(line):
findings.add(f"{prefix}: private application path")
if PRIVATE_KEY.search(line):
findings.add(f"{prefix}: private-key block")
for match in EMAIL.finditer(line):
domain = match.group(1).lower()
if not domain.endswith(("example.com", "example.net", "example.org")):
findings.add(f"{prefix}: non-example email address")
for match in SECRET_ASSIGNMENT.finditer(line):
value = match.group(1)
if not is_placeholder(value):
findings.add(f"{prefix}: non-placeholder secret assignment")
for match in AUTHORIZATION.finditer(line):
value = match.group(1)
if not is_placeholder(value):
findings.add(f"{prefix}: authorization material")
for candidate in OPAQUE.findall(line):
if is_placeholder(candidate):
continue
# Commit hashes and content digests are still identifiers; require a label.
labelled_hash = re.search(
r"(?i)\b(?:sha(?:1|256|512)|digest|commit|checksum|example[_-]?hash)\b",
line,
)
if labelled_hash and re.fullmatch(r"[0-9a-fA-F]{32,128}", candidate):
continue
if entropy(candidate) >= 3.5:
findings.add(f"{prefix}: opaque high-entropy value")
break
if findings:
print("Public-safety scan failed:")
for finding in sorted(findings):
print(f"- {finding}")
return 1
print(f"Public-safety scan passed: {len(files)} tracked files checked")
return 0
if __name__ == "__main__":
sys.exit(main())