Sync network redesign repo state and cutover artifacts
This commit is contained in:
@@ -6,9 +6,9 @@ Full homelab stack as of 2026-05-09. All 6 expansion phases are complete and dep
|
||||
|
||||
| Server | OS | IP | Role |
|
||||
|--------|----|----|------|
|
||||
| **PlausibleDeniability (PD)** | TrueNAS Scale 25.10.2.1 | 10.5.1.6 | Primary Docker host — all compose stacks |
|
||||
| **Serenity** | Unraid 7.2.4 | 10.5.1.5 | NAS, ARR stack, CPU reranker |
|
||||
| **N.O.M.A.D.** | Ubuntu 25.10 | 10.5.1.16 | Offline knowledge, game servers, and standalone local services |
|
||||
| **PlausibleDeniability (PD)** | TrueNAS Scale 25.10.2.1 | 10.5.30.6 | Primary Docker host — all compose stacks |
|
||||
| **Serenity** | Unraid 7.2.4 | 10.5.30.5 | NAS, ARR stack, CPU reranker |
|
||||
| **N.O.M.A.D.** | Ubuntu 25.10 | 10.5.30.7 | Offline knowledge, game servers, and standalone local services |
|
||||
| **Rocinante** | (bare metal) | 10.5.1.112 | Heavy Ollama models (RTX 4090) |
|
||||
|
||||
## Network
|
||||
@@ -24,7 +24,7 @@ Full homelab stack as of 2026-05-09. All 6 expansion phases are complete and dep
|
||||
- **Homepage** (3300) — dashboard / service index
|
||||
- **Uptime Kuma** (3001) — monitoring and alerting
|
||||
- **Gotify** (8443) — push notifications
|
||||
- **Gitea** (3002 web / 2222 SSH) — self-hosted git; internal SSH remote uses `ssh://git@10.5.1.6:2222/<owner>/<repo>.git`
|
||||
- **Gitea** (3002 web / 2222 SSH) — self-hosted git; internal SSH remote uses `ssh://git@10.5.30.6:2222/<owner>/<repo>.git`
|
||||
- **Dockhand** (3230) — Docker management UI
|
||||
|
||||
### AI Layer
|
||||
@@ -61,4 +61,4 @@ Full homelab stack as of 2026-05-09. All 6 expansion phases are complete and dep
|
||||
3. **External Docker networks** — `ai-services` bridges all AI containers across compose files; `ix-databases_shared-databases` provides shared DB access
|
||||
4. **NFS for Immich** — photo library lives on Serenity's large array; PD mounts on demand (not at boot)
|
||||
5. **TrueNAS manages Docker** — never use `systemctl restart docker` on PD; use `docker compose` or `docker restart` for individual containers
|
||||
6. **Credential hygiene** — Gitea pushes from PD/NOMAD use internal SSH (`ssh://git@10.5.1.6:2222/<owner>/<repo>.git`); GitHub remote uses SSH key (`/root/.ssh/id_gitea`)
|
||||
6. **Credential hygiene** — Gitea pushes from PD/NOMAD use internal SSH (`ssh://git@10.5.30.6:2222/<owner>/<repo>.git`); GitHub remote uses SSH key (`/root/.ssh/id_gitea`)
|
||||
|
||||
@@ -2,7 +2,7 @@
|
||||
|
||||
## Serenity (NAS / Media Ingestion)
|
||||
- **OS:** Unraid 7.2.4
|
||||
- **IP:** 10.5.1.5 | Tailscale: 100.94.87.79
|
||||
- **IP:** 10.5.30.5 | Tailscale: 100.94.87.79
|
||||
- **CPU:** Dual Xeon X5690 (6c/12t each — 24 threads total)
|
||||
- **RAM:** 102GB
|
||||
- **GPU:** None (Matrox IPMI only)
|
||||
@@ -51,7 +51,7 @@
|
||||
|
||||
## N.O.M.A.D. (Offline Knowledge + Game Servers)
|
||||
- **OS:** Ubuntu 25.10 (bare metal)
|
||||
- **IP:** 10.5.1.16
|
||||
- **IP:** 10.5.30.7
|
||||
- **CPU:** Intel Core i7-4770K @ 3.50GHz (4c/8t, up to 3.9GHz)
|
||||
- **RAM:** 32GB DDR3 1600MHz (4×8GB)
|
||||
- **GPU:** NVIDIA GeForce GTX 1080 (8GB VRAM)
|
||||
|
||||
@@ -12,9 +12,9 @@
|
||||
- **Main LAN:** 10.5.1.x (Trusted VLAN)
|
||||
- **Management VLAN:** 10.5.0.x
|
||||
- **Other VLANs:** Cameras, Guest, IoT / Old IoT
|
||||
- **Serenity IP:** 10.5.1.5
|
||||
- **N.O.M.A.D. IP:** 10.5.1.16
|
||||
- **PlausibleDeniability:** 10.5.1.6 (static on LAN)
|
||||
- **Serenity IP:** 10.5.30.5
|
||||
- **N.O.M.A.D. IP:** 10.5.30.7
|
||||
- **PlausibleDeniability:** 10.5.30.6 (static on LAN)
|
||||
|
||||
## Remote Access
|
||||
- **Pangolin VPS + Newt:** Services exposed as subdomains via reverse proxy. Newt agents run on PD and N.O.M.A.D.
|
||||
|
||||
@@ -20,7 +20,7 @@ All homelab services, their hosts, ports, and current status.
|
||||
| Dockhand | 3230 | http://pd:3230 | ✅ Active |
|
||||
| Uptime Kuma | 3001 | http://pd:3001 | ✅ Active |
|
||||
| Gotify | 8443 | https://gotify.paccoco.com | ✅ Active |
|
||||
| Gitea | 3002 / 2222 | http://pd:3002 / http://10.5.1.6:3002 (SSH: `ssh://git@10.5.1.6:2222/<owner>/<repo>.git`) | ✅ Active |
|
||||
| Gitea | 3002 / 2222 | http://pd:3002 / http://10.5.30.6:3002 (SSH: `ssh://git@10.5.30.6:2222/<owner>/<repo>.git`) | ✅ Active |
|
||||
| Shlink | 8087 | http://pd:8087 | ✅ Active |
|
||||
| Shlink Web Client | 8088 | http://pd:8088 | ✅ Active |
|
||||
| RackPeek | 8283 | http://pd:8283 | ✅ Active |
|
||||
@@ -33,12 +33,12 @@ All homelab services, their hosts, ports, and current status.
|
||||
| LiteLLM | PD | 4000 | ✅ Active |
|
||||
| OpenWebUI | PD | 8282 | ✅ Active |
|
||||
| Qdrant | PD | 6333 (HTTP) / 6334 (gRPC) | ✅ Active |
|
||||
| Whisper (faster-whisper, CPU) | N.O.M.A.D. (10.5.1.16) | 8786 | ✅ Active |
|
||||
| Whisper (faster-whisper, CPU) | N.O.M.A.D. (10.5.30.7) | 8786 | ✅ Active |
|
||||
| Whisper (faster-whisper, CUDA large-v3) | Rocinante (10.5.1.112) | 8787 | ⚠️ Unreachable from N.O.M.A.D. during 2026-05-21 Doris validation |
|
||||
| SearXNG | PD | 8888 | ✅ Active |
|
||||
| Ollama — light tier | PD (10.5.1.6) | 11434 | ✅ Active |
|
||||
| Ollama — light tier | PD (10.5.30.6) | 11434 | ✅ Active |
|
||||
| Ollama — heavy tier | Rocinante (10.5.1.112) | 11434 | ⚠️ Unreachable from N.O.M.A.D. during 2026-05-21 Doris validation |
|
||||
| Reranker (TEI) | Serenity (10.5.1.5) | 9787 | ✅ Active |
|
||||
| Reranker (TEI) | Serenity (10.5.30.5) | 9787 | ✅ Active |
|
||||
|
||||
OpenClaw is no longer treated as an active PD service in the operator inventory. Only helper-artifact references remain under the school-intake planning docs.
|
||||
|
||||
@@ -87,7 +87,7 @@ OpenClaw is no longer treated as an active PD service in the operator inventory.
|
||||
|
||||
| Service | Host | Port | Status |
|
||||
|---------|------|------|--------|
|
||||
| MeshCore to MQTT relay | N.O.M.A.D. (10.5.1.16) | outbound MQTT only | ✅ Running as Docker container `mctomqtt` (`mctomqtt:latest`); config bind-mounted from `/etc/mctomqtt` |
|
||||
| MeshCore to MQTT relay | N.O.M.A.D. (10.5.30.7) | outbound MQTT only | ✅ Running as Docker container `mctomqtt` (`mctomqtt:latest`); config bind-mounted from `/etc/mctomqtt` |
|
||||
|
||||
## Mapping / Geo (PD)
|
||||
|
||||
@@ -95,11 +95,11 @@ OpenClaw is no longer treated as an active PD service in the operator inventory.
|
||||
|---------|------|--------|
|
||||
| TileServer GL | 8082 | ✅ Active |
|
||||
|
||||
## Game Servers (N.O.M.A.D. — 10.5.1.16)
|
||||
## Game Servers (N.O.M.A.D. — 10.5.30.7)
|
||||
|
||||
| Service | Port | URL | Status | Notes |
|
||||
|---------|------|-----|--------|-------|
|
||||
| Pelican Panel | 80 / 443 | https://panel.paccoco.com | ✅ Active | For local health/probe use `http://10.5.1.16:8080`; local TLS path is self-signed and `panel.paccoco.com` resolves to `127.0.0.1` on N.O.M.A.D. |
|
||||
| Pelican Panel | 80 / 443 | https://panel.paccoco.com | ✅ Active | For local health/probe use `http://10.5.30.7:8080`; local TLS path is self-signed and `panel.paccoco.com` resolves to `127.0.0.1` on N.O.M.A.D. |
|
||||
| Wings Node API | 8443 / 443 | https://node1.paccoco.com | ✅ Active | `401` from the public endpoint is expected and means the daemon is reachable/auth-protected |
|
||||
| Minecraft Server 25565 | 25565/tcp+udp | managed via Pelican | ✅ Active | Wings-managed Java server |
|
||||
| Minecraft Server 25566 | 25566/tcp+udp | managed via Pelican | ✅ Active | Wings-managed Java/NeoForge server |
|
||||
|
||||
@@ -28,7 +28,7 @@ Deploy the PD → Serenity backup runner from the real compose path on PD:
|
||||
## Suggested `.env` values to review
|
||||
|
||||
```dotenv
|
||||
SERENITY_BACKUP_HOST=root@10.5.1.5
|
||||
SERENITY_BACKUP_HOST=root@10.5.30.5
|
||||
SERENITY_BACKUP_ROOT=/mnt/user/backups/plausible-deniability
|
||||
PD_APPDATA_ROOT=/mnt/docker-ssd/docker/appdata
|
||||
PD_DATABASES_ROOT=/mnt/docker-ssd/docker/databases
|
||||
@@ -79,7 +79,7 @@ Quarterly restore verification:
|
||||
```bash
|
||||
tail -100 /mnt/tank/docker/backups/pd-backups.log
|
||||
ls -lh /mnt/tank/docker/backups/db-dumps
|
||||
ssh root@10.5.1.5 'find /mnt/user/backups/plausible-deniability -maxdepth 2 -type d | sort | head -40'
|
||||
ssh root@10.5.30.5 'find /mnt/user/backups/plausible-deniability -maxdepth 2 -type d | sort | head -40'
|
||||
```
|
||||
|
||||
## Restore verification
|
||||
|
||||
@@ -63,7 +63,7 @@ All `.env` files live alongside their `docker-compose.yaml`:
|
||||
| | |
|
||||
|---|---|
|
||||
| **Gitea repo** | `https://gitea.paccoco.com/fizzlepoof/homelab-secrets` (private) |
|
||||
| **SSH remote** | `ssh://git@10.5.1.6:2222/fizzlepoof/homelab-secrets.git` |
|
||||
| **SSH remote** | `ssh://git@10.5.30.6:2222/fizzlepoof/homelab-secrets.git` |
|
||||
| **Local path on PD** | `/mnt/docker-ssd/docker/secrets/` |
|
||||
| **Symmetric key location** | `/mnt/docker-ssd/docker/secrets/.git-crypt-secrets.key` (if copied there) or retrieve from password manager |
|
||||
| **Key backup** | Password manager + `C:\Users\Fizzlepoof\Downloads\.git-crypt-secrets.key` |
|
||||
@@ -150,7 +150,7 @@ sudo cp /root/bin/git-crypt /mnt/docker-ssd/bin/git-crypt
|
||||
|
||||
```bash
|
||||
# Install git-crypt first (see above), then:
|
||||
git clone ssh://git@10.5.1.6:2222/fizzlepoof/homelab-secrets.git /mnt/docker-ssd/docker/secrets
|
||||
git clone ssh://git@10.5.30.6:2222/fizzlepoof/homelab-secrets.git /mnt/docker-ssd/docker/secrets
|
||||
cd /mnt/docker-ssd/docker/secrets
|
||||
git-crypt unlock /path/to/.git-crypt-secrets.key
|
||||
```
|
||||
@@ -166,7 +166,7 @@ On a fresh PD after reinstalling TrueNAS:
|
||||
1. **Install git-crypt** (Option 1 above — docker is available immediately after TrueNAS install)
|
||||
2. **Restore secrets repo:**
|
||||
```bash
|
||||
git clone ssh://git@10.5.1.6:2222/fizzlepoof/homelab-secrets.git /mnt/docker-ssd/docker/secrets
|
||||
git clone ssh://git@10.5.30.6:2222/fizzlepoof/homelab-secrets.git /mnt/docker-ssd/docker/secrets
|
||||
cd /mnt/docker-ssd/docker/secrets
|
||||
git-crypt unlock /path/to/.git-crypt-secrets.key
|
||||
```
|
||||
@@ -178,7 +178,7 @@ On a fresh PD after reinstalling TrueNAS:
|
||||
```
|
||||
4. **Clone the main stacks repo:**
|
||||
```bash
|
||||
git clone ssh://git@10.5.1.6:2222/fizzlepoof/truenas-stacks.git /mnt/docker-ssd/docker/compose
|
||||
git clone ssh://git@10.5.30.6:2222/fizzlepoof/truenas-stacks.git /mnt/docker-ssd/docker/compose
|
||||
```
|
||||
5. **Redeploy stacks** per `DEPLOYMENT_CHECKLIST.md`
|
||||
6. **After any future live `.env` edit**, immediately run:
|
||||
|
||||
@@ -7,8 +7,8 @@ NOMAD exception: standalone live deployments belong under `/opt/<service>` rathe
|
||||
|
||||
Remotes:
|
||||
- GitHub: `git@github.com:Paccoco/truenas-stacks.git`
|
||||
- Gitea web: `http://10.5.1.6:3002/fizzlepoof/truenas-stacks`
|
||||
- Gitea SSH: `ssh://git@10.5.1.6:2222/fizzlepoof/truenas-stacks.git`
|
||||
- Gitea web: `http://10.5.30.6:3002/fizzlepoof/truenas-stacks`
|
||||
- Gitea SSH: `ssh://git@10.5.30.6:2222/fizzlepoof/truenas-stacks.git`
|
||||
|
||||
Use the internal Gitea SSH remote from PD/NOMAD. Port `2222` is SSH, not HTTP.
|
||||
|
||||
@@ -144,7 +144,7 @@ After changing a healthcheck, must `down` then `up` (not just restart).
|
||||
cd /mnt/docker-ssd/docker/compose
|
||||
git add .
|
||||
git commit -m "description"
|
||||
git push ssh://git@10.5.1.6:2222/fizzlepoof/truenas-stacks.git main
|
||||
git push ssh://git@10.5.30.6:2222/fizzlepoof/truenas-stacks.git main
|
||||
```
|
||||
|
||||
All `.env` files are gitignored. Only compose files, `.env.example`, and init scripts are committed.
|
||||
|
||||
@@ -94,7 +94,7 @@ sudo docker logs --tail 60 kima
|
||||
|
||||
## Post-deploy checks
|
||||
|
||||
1. Open `http://10.5.1.6:3333`
|
||||
1. Open `http://10.5.30.6:3333`
|
||||
2. Confirm the app starts and initial setup loads
|
||||
3. Confirm `/music` is visible to Kima
|
||||
4. Confirm Kima creates and uses `/data` persistence correctly
|
||||
@@ -103,7 +103,7 @@ sudo docker logs --tail 60 kima
|
||||
|
||||
Verified on 2026-05-15:
|
||||
- container `kima` running and healthy
|
||||
- `http://10.5.1.6:3333/api/health` returned status OK
|
||||
- `http://10.5.30.6:3333/api/health` returned status OK
|
||||
- live mount narrowed to `/mnt/unraid/data/media/music:/music`
|
||||
|
||||
## Optional integrations / secrets
|
||||
|
||||
@@ -1,23 +1,25 @@
|
||||
# Pi-hole Deployment Plan
|
||||
|
||||
**Status:** PD primary and NOMAD backup are live and working; DNS clients are pointed at VIP `10.5.1.53`. RPi4 remains optional future BACKUP2 work.
|
||||
**Status:** Historical Trusted-VLAN design. PD primary is still valid, but after NOMAD moved to Servers (`10.5.30.7`) its participation in the old Trusted-side VIP `10.5.30.53` was disabled to restore correct routing. Do not reuse this document for NOMAD as-is until a same-subnet HA redesign exists.
|
||||
**Architecture:** 3-node HA cluster with Keepalived VIP, Unbound recursive DNS, Nebula Sync
|
||||
|
||||
> **Warning:** The examples below describe the original Trusted-side plan where NOMAD was `10.5.30.7` and shared VIP `10.5.30.53/24`. They are retained as historical deployment notes, not the current NOMAD runtime truth.
|
||||
|
||||
---
|
||||
|
||||
## Architecture Overview
|
||||
|
||||
```
|
||||
[All LAN clients]
|
||||
↓ DNS: 10.5.1.53 (Virtual IP — floats to whichever node is MASTER)
|
||||
↓ DNS: 10.5.30.53 (Virtual IP — floats to whichever node is MASTER)
|
||||
↓
|
||||
┌─────┴──────────────────────────────────┐
|
||||
│ Keepalived VIP │
|
||||
│ 10.5.1.53/24 │
|
||||
│ 10.5.30.53/24 │
|
||||
└──────┬──────────────┬──────────────────┘
|
||||
│ │ │
|
||||
PD (MASTER) N.O.M.A.D. (BACKUP1) RPi4 (BACKUP2)
|
||||
10.5.1.6 10.5.1.16 10.5.1.X
|
||||
10.5.30.6 10.5.30.7 10.5.1.X
|
||||
priority 100 priority 90 priority 80
|
||||
Pi-hole :53 Pi-hole :53 Pi-hole :53
|
||||
Unbound :5335 Unbound :5335 Unbound :5335
|
||||
@@ -28,11 +30,11 @@
|
||||
(recursive — no third-party upstream)
|
||||
```
|
||||
|
||||
- **Keepalived** floats the VIP `10.5.1.53` to whichever node is alive with the highest priority
|
||||
- **Keepalived** floats the VIP `10.5.30.53` to whichever node is alive with the highest priority
|
||||
- **Unbound** on each node acts as the local recursive resolver (replaces Cloudflare/Quad9)
|
||||
- **Pi-hole** on each node forwards upstream queries to its local Unbound (`127.0.0.1#5335`)
|
||||
- **Nebula Sync** (running on PD) syncs all Pi-hole configs from primary → replicas daily
|
||||
- Current live replica target is NOMAD at `10.5.1.16:8954`
|
||||
- Current live replica target is NOMAD at `10.5.30.7:8954`
|
||||
|
||||
All nodes use `network_mode: host` for Pi-hole, Unbound, and Keepalived — this is required for Keepalived to manipulate the VIP and for Pi-hole to bind cleanly to port 53.
|
||||
|
||||
@@ -40,7 +42,7 @@ All nodes use `network_mode: host` for Pi-hole, Unbound, and Keepalived — this
|
||||
|
||||
## Pre-Deploy: Network Planning
|
||||
|
||||
1. **Pick a free static IP for the VIP** — `10.5.1.53` is recommended (memorable for DNS). Reserve it in UniFi so DHCP never assigns it.
|
||||
1. **Pick a free static IP for the VIP** — `10.5.30.53` is recommended (memorable for DNS). Reserve it in UniFi so DHCP never assigns it.
|
||||
2. **Find the LAN interface name** on each node:
|
||||
```bash
|
||||
# PD / RPi4:
|
||||
@@ -171,7 +173,7 @@ services:
|
||||
volumes:
|
||||
- ./keepalived.conf:/container/environment/01-custom/keepalived.conf
|
||||
environment:
|
||||
KEEPALIVED_VIRTUAL_IPS: "#PYTHON2BASH:['10.5.1.53/24']"
|
||||
KEEPALIVED_VIRTUAL_IPS: "#PYTHON2BASH:['10.5.30.53/24']"
|
||||
```
|
||||
|
||||
### keepalived.conf (PD — MASTER)
|
||||
@@ -195,7 +197,7 @@ vrrp_instance VI_DNS {
|
||||
}
|
||||
|
||||
virtual_ipaddress {
|
||||
10.5.1.53/24
|
||||
10.5.30.53/24
|
||||
}
|
||||
}
|
||||
```
|
||||
@@ -323,7 +325,7 @@ vrrp_instance VI_DNS {
|
||||
}
|
||||
|
||||
virtual_ipaddress {
|
||||
10.5.1.53/24
|
||||
10.5.30.53/24
|
||||
}
|
||||
}
|
||||
```
|
||||
@@ -409,7 +411,7 @@ vrrp_instance VI_DNS {
|
||||
}
|
||||
|
||||
virtual_ipaddress {
|
||||
10.5.1.53/24
|
||||
10.5.30.53/24
|
||||
}
|
||||
}
|
||||
```
|
||||
@@ -434,7 +436,7 @@ sudo systemctl start keepalived
|
||||
restart: unless-stopped
|
||||
environment:
|
||||
PRIMARY: "http://127.0.0.1:8953|${PIHOLE_PASSWORD}"
|
||||
REPLICAS: "http://10.5.1.16:8954|${PIHOLE_PASSWORD_NOMAD},http://10.5.1.X:8955|${PIHOLE_PASSWORD_RPI}"
|
||||
REPLICAS: "http://10.5.30.7:8954|${PIHOLE_PASSWORD_NOMAD},http://10.5.1.X:8955|${PIHOLE_PASSWORD_RPI}"
|
||||
RUN_GRAVITY: "true"
|
||||
FULL_SYNC: "false"
|
||||
CRON: "0 3 * * *"
|
||||
@@ -461,7 +463,7 @@ sudo systemctl start keepalived
|
||||
Point **only the VIP** as DNS:
|
||||
|
||||
1. UniFi Network → Settings → Networks → **LAN** → DHCP Name Server: **Manual**
|
||||
2. **DNS Server 1:** `10.5.1.53` (VIP — floats between nodes)
|
||||
2. **DNS Server 1:** `10.5.30.53` (VIP — floats between nodes)
|
||||
3. **DNS Server 2:** *(leave blank or set a public fallback like `1.1.1.1` only if you want internet DNS as last resort)*
|
||||
4. Save and renew DHCP leases
|
||||
|
||||
@@ -472,9 +474,9 @@ Point **only the VIP** as DNS:
|
||||
|
||||
## Current live endpoints
|
||||
|
||||
- **PD admin UI:** `http://10.5.1.6:8953/admin/`
|
||||
- **NOMAD admin UI:** `http://10.5.1.16:8954/admin/`
|
||||
- **VIP for DNS clients:** `10.5.1.53`
|
||||
- **PD admin UI:** `http://10.5.30.6:8953/admin/`
|
||||
- **NOMAD admin UI:** `http://10.5.30.7:8954/admin/`
|
||||
- **VIP for DNS clients:** `10.5.30.53`
|
||||
|
||||
## Post-Deploy
|
||||
|
||||
@@ -482,11 +484,11 @@ Point **only the VIP** as DNS:
|
||||
|
||||
```bash
|
||||
# Check which node holds the VIP
|
||||
ip addr show | grep 10.5.1.53 # Run on each node — only MASTER shows it
|
||||
ip addr show | grep 10.5.30.53 # Run on each node — only MASTER shows it
|
||||
|
||||
# Test DNS through VIP
|
||||
nslookup google.com 10.5.1.53
|
||||
nslookup doubleclick.net 10.5.1.53 # Should return 0.0.0.0 (blocked)
|
||||
nslookup google.com 10.5.30.53
|
||||
nslookup doubleclick.net 10.5.30.53 # Should return 0.0.0.0 (blocked)
|
||||
```
|
||||
|
||||
### Test failover
|
||||
@@ -495,7 +497,7 @@ nslookup doubleclick.net 10.5.1.53 # Should return 0.0.0.0 (blocked)
|
||||
# On PD, bring down keepalived
|
||||
sudo docker stop keepalived
|
||||
# Wait ~3 seconds, then check from another machine:
|
||||
nslookup google.com 10.5.1.53 # Should still work — N.O.M.A.D. took over VIP
|
||||
nslookup google.com 10.5.30.53 # Should still work — N.O.M.A.D. took over VIP
|
||||
# Restore
|
||||
sudo docker start keepalived
|
||||
```
|
||||
@@ -516,8 +518,8 @@ Add in Pi-hole (primary) → Local DNS → DNS Records:
|
||||
|
||||
| Domain | IP |
|
||||
|--------|----|
|
||||
| pd.lan | 10.5.1.6 |
|
||||
| nomad.lan | 10.5.1.16 |
|
||||
| pd.lan | 10.5.30.6 |
|
||||
| nomad.lan | 10.5.30.7 |
|
||||
| rocinante.lan | 10.5.1.112 |
|
||||
| rpi4.lan | 10.5.1.X |
|
||||
|
||||
@@ -527,8 +529,8 @@ Nebula Sync will propagate these to all replicas.
|
||||
|
||||
| Subdomain | Target |
|
||||
|-----------|--------|
|
||||
| `pihole.paccoco.com` | `http://10.5.1.6:8953` (primary) |
|
||||
| `pihole2.paccoco.com` | `http://10.5.1.16:8954` (N.O.M.A.D.) |
|
||||
| `pihole.paccoco.com` | `http://10.5.30.6:8953` (primary) |
|
||||
| `pihole2.paccoco.com` | `http://10.5.30.7:8954` (N.O.M.A.D.) |
|
||||
|
||||
Use `/admin/` on the public hostname for the Pi-hole admin UI path.
|
||||
|
||||
|
||||
@@ -88,7 +88,7 @@ Generated: 2026-05-09
|
||||
### 12 — Pi-hole Stats Digest
|
||||
**Trigger:** Schedule (daily at 8 AM)
|
||||
**What it does:**
|
||||
1. Query Pi-hole API on PD (`http://10.5.1.6:8953/api/stats/summary`)
|
||||
1. Query Pi-hole API on PD (`http://10.5.30.6:8953/api/stats/summary`)
|
||||
2. Query secondary Pi-hole on Serenity
|
||||
3. LiteLLM summarizes notable blocked domains and compares primary vs secondary query load
|
||||
4. Push digest to Gotify (Pi-hole app) with 24h block count, top blocked domains, and any unusual spikes
|
||||
@@ -136,7 +136,7 @@ Generated: 2026-05-09
|
||||
### 16 — NFS Mount Watchdog + Auto-Heal
|
||||
**Trigger:** Schedule (every 5 min)
|
||||
**What it does:**
|
||||
1. SSH into PD (10.5.1.6) and probe each expected NFS mount path with `mountpoint -q`:
|
||||
1. SSH into PD (10.5.30.6) and probe each expected NFS mount path with `mountpoint -q`:
|
||||
- `/mnt/unraid/data/media` (Plex, Audiobookshelf)
|
||||
- `/mnt/unraid/data/photos` (Immich)
|
||||
- Any other mounts defined in the mount script
|
||||
@@ -153,7 +153,7 @@ Generated: 2026-05-09
|
||||
- Escalation: priority 9, list which paths are still unavailable
|
||||
|
||||
**Implementation notes:**
|
||||
- Uses n8n SSH node connecting to `10.5.1.6` with an SSH credential (key-based auth from n8n container)
|
||||
- Uses n8n SSH node connecting to `10.5.30.6` with an SSH credential (key-based auth from n8n container)
|
||||
- SSH node runs a single compound command: `mountpoint -q /mnt/unraid/data/media && echo MEDIA_OK || echo MEDIA_MISSING` etc.
|
||||
- Parse output in a Code node to determine which mounts failed
|
||||
- Use separate SSH nodes for: (1) running the mount script, (2) restarting containers — keeps the flow readable
|
||||
|
||||
@@ -5,7 +5,7 @@
|
||||
- [x] Regenerate N.O.M.A.D. Newt secret (was exposed in chat) — completed 2026-05-15
|
||||
|
||||
## Infrastructure
|
||||
- [x] Deploy Pi-hole HA on PD + N.O.M.A.D. with VIP `10.5.1.53` — completed 2026-05-15; see docs/planning/DEPLOY_PIHOLE.md
|
||||
- [x] Deploy Pi-hole HA on PD + N.O.M.A.D. with VIP `10.5.30.53` — completed 2026-05-15; historical Trusted-VLAN design, later superseded after NOMAD moved to Servers (`10.5.30.7`) and its Trusted-side Keepalived role was disabled pending a same-subnet HA redesign; see docs/planning/DEPLOY_PIHOLE.md
|
||||
- PD primary live at `/mnt/docker-ssd/docker/compose/pihole`
|
||||
- NOMAD backup live at `/opt/pihole-nomad`
|
||||
- RPi4 remains optional future BACKUP2 work if you want a third node later
|
||||
|
||||
@@ -34,9 +34,9 @@ See the individual docs in this repo for detailed information:
|
||||
### Three Servers
|
||||
| Server | Role | OS | IP |
|
||||
|--------|------|----|----|
|
||||
| Serenity | NAS / Media ingestion | Unraid 7.2.4 | 10.5.1.5 |
|
||||
| PlausibleDeniability | Primary Docker host | TrueNAS Scale 25.10.2.1 | 10.5.1.6 |
|
||||
| N.O.M.A.D. | Offline knowledge + game servers | Ubuntu 25.10 | 10.5.1.16 |
|
||||
| Serenity | NAS / Media ingestion | Unraid 7.2.4 | 10.5.30.5 |
|
||||
| PlausibleDeniability | Primary Docker host | TrueNAS Scale 25.10.2.1 | 10.5.30.6 |
|
||||
| N.O.M.A.D. | Offline knowledge + game servers | Ubuntu 25.10 | 10.5.30.7 |
|
||||
|
||||
### Deployment Pattern (PD)
|
||||
```bash
|
||||
|
||||
@@ -4,7 +4,7 @@ Offline survival/knowledge server and game server host.
|
||||
|
||||
## Specs
|
||||
- **OS:** Ubuntu 25.10 (bare metal)
|
||||
- **IP:** 10.5.1.16
|
||||
- **IP:** 10.5.30.7
|
||||
- **CPU:** Intel Core i7-4770K @ 3.50GHz (4c/8t)
|
||||
- **RAM:** 32GB DDR3 1600MHz
|
||||
- **GPU:** NVIDIA GeForce GTX 1080 (8GB VRAM)
|
||||
@@ -309,7 +309,7 @@ Removing them should not be treated as a Project N.O.M.A.D. change or a Wings/Pe
|
||||
- `panel.paccoco.com` → `127.0.0.1` in `/etc/hosts` for NAT loopback
|
||||
- nginx on 80 and 443 (self-signed cert) for local Wings → Panel
|
||||
- Wings uses `--ignore-certificate-errors` for self-signed cert
|
||||
- For local operator-safe Pelican health checks, use `http://10.5.1.16:8080` instead of the self-signed TLS path; it redirects cleanly to `/home` without tripping certificate validation.
|
||||
- For local operator-safe Pelican health checks, use `http://10.5.30.7:8080` instead of the self-signed TLS path; it redirects cleanly to `/home` without tripping certificate validation.
|
||||
- Pangolin resource `node1.paccoco.com` has auth disabled
|
||||
|
||||
## Known Quirks
|
||||
|
||||
@@ -33,7 +33,7 @@ RTX 2080 Ti used by:
|
||||
- immich-ml (CUDA face recognition)
|
||||
- Ollama light-tier inference for the main AI stack and local Honcho backend
|
||||
|
||||
> **Note:** Whisper moved to N.O.M.A.D. (10.5.1.16:8786) as CPU inference — avoids VRAM conflict with Ollama. See [NOMAD.md](NOMAD.md).
|
||||
> **Note:** Whisper moved to N.O.M.A.D. (10.5.30.7:8786) as CPU inference — avoids VRAM conflict with Ollama. See [NOMAD.md](NOMAD.md).
|
||||
|
||||
Recommended models for 11GB VRAM:
|
||||
| Model | VRAM | Best for |
|
||||
@@ -45,8 +45,8 @@ Recommended models for 11GB VRAM:
|
||||
| `phi4` | ~8GB | Analytical tasks |
|
||||
|
||||
Current light-tier notes:
|
||||
- PD Ollama is the active light-tier endpoint for LiteLLM on `http://10.5.1.6:11434`
|
||||
- Local Honcho currently points at PD Ollama over LAN (`http://10.5.1.6:11434/v1`) instead of using a separate local model host
|
||||
- PD Ollama is the active light-tier endpoint for LiteLLM on `http://10.5.30.6:11434`
|
||||
- Local Honcho currently points at PD Ollama over LAN (`http://10.5.30.6:11434/v1`) instead of using a separate local model host
|
||||
- Current Honcho routing split keeps embeddings on `nomic-embed-text:latest`, cheap/default paths on `qwen2.5:7b`, and deeper high/max paths on `qwen2.5:14b`
|
||||
|
||||
## Remote Access
|
||||
|
||||
@@ -6,7 +6,7 @@ NAS, media ingestion, and CPU-based AI inference.
|
||||
|
||||
| Component | Details |
|
||||
|-----------|---------|
|
||||
| **IP (LAN)** | 10.5.1.5 |
|
||||
| **IP (LAN)** | 10.5.30.5 |
|
||||
| **IP (Tailscale)** | 100.94.87.79 |
|
||||
| **OS** | Unraid 7.2.4 |
|
||||
| **CPU** | Dual Intel Xeon X5690 (6c/12t each — 24 threads total) |
|
||||
@@ -44,7 +44,7 @@ Text Embeddings Inference reranker for OpenWebUI RAG pipeline.
|
||||
| **Port** | `9787` |
|
||||
| **Volume** | `/mnt/user/appdata/reranker` |
|
||||
| **LiteLLM model name** | `reranker` |
|
||||
| **API base** | `http://10.5.1.5:9787` |
|
||||
| **API base** | `http://10.5.30.5:9787` |
|
||||
|
||||
> Moved from PlausibleDeniability (port 8787) to Serenity (port 9787) on 2026-05-09.
|
||||
> CPU-only inference is acceptable for RAG reranking workloads.
|
||||
|
||||
Reference in New Issue
Block a user