diff --git a/DEPLOYMENT_HANDOFF.md b/DEPLOYMENT_HANDOFF.md index efade28..16f55ca 100644 --- a/DEPLOYMENT_HANDOFF.md +++ b/DEPLOYMENT_HANDOFF.md @@ -7,6 +7,7 @@ Phase 2 (AI stack) is fully deployed and verified on PlausibleDeniability (10.5. ## What's Done ### Phase 2 — AI Stack (COMPLETE) +Historical note: this handoff reflects the original PD-first AI deployment snapshot, not the current Honcho routing policy. All 6 containers running and verified on PD (10.5.30.6): - **Ollama:** port 11434 — qwen2.5:14b loaded - **OpenWebUI:** port 8282 (→8080) diff --git a/HOMELAB_BUILDOUT_PLAN.md b/HOMELAB_BUILDOUT_PLAN.md index 37b6bd5..222b6b2 100644 --- a/HOMELAB_BUILDOUT_PLAN.md +++ b/HOMELAB_BUILDOUT_PLAN.md @@ -575,7 +575,7 @@ 575|Trigger: Cron node (every Sunday at 10:00 AM) 576|Step 1: Postgres node → SELECT watch history for past 7 days 577|Step 2: Format data as structured text - 578|Step 3: HTTP Request → Ollama API (POST to PD's qwen2.5:14b) + 578|Step 3: HTTP Request → Ollama API (POST to the local/default automation model host, preferably N.O.M.A.D.-local rather than PD) 579| - Prompt: "Summarize this week's viewing in a fun digest: {data}" 580|Step 4: HTTP Request → Gotify (send digest) 581|``` @@ -1093,9 +1093,9 @@ 1388| 1389|LiteLLM sits in front of your three Ollama instances and presents a single OpenAI-compatible API at `http://litellm:4000`. It handles: 1390| - 1391|- **Model routing** — requests for `qwen3:32b` go to ROCINANTE, `qwen2.5:14b` goes to PD, `phi4` goes to N.O.M.A.D. - 1392|- **Failover** — if ROCINANTE is offline, LiteLLM can fall back to PD automatically - 1393|- **Load balancing** — distribute requests across instances running the same model + - **Model routing** — current reliable default is N.O.M.A.D.-local Ollama for background/automation paths, with PD kept available for shared light-tier use and Rocinante treated as optional/manual heavy capacity. + - **Failover** — avoid designs that require Rocinante to be online; degrade to N.O.M.A.D.-local or explicitly operator-invoked PD paths instead of automatic dependence on a personal PC. + - **Load balancing** — distribute requests across instances running the same model when you have multiple dependable backends, but do not assume Rocinante qualifies as one. 1394|- **Usage tracking** — logs token counts, latency, and costs per model/user via its built-in database 1395| 1396|### 7.2 — Scaffold Directories @@ -1621,17 +1621,23 @@ 1916|{ 1917| "models": [ 1918| { - 1919| "title": "PD - qwen2.5:14b", + 1919| "title": "N.O.M.A.D. - qwen2.5:3b", 1920| "provider": "ollama", - 1921| "model": "qwen2.5:14b", - 1922| "apiBase": "http://PD_TAILSCALE_IP:11434" + 1921| "model": "qwen2.5:3b", + 1922| "apiBase": "http://NOMAD_LOCAL_OR_TAILSCALE_IP:11434" 1923| }, 1924| { - 1925| "title": "ROCINANTE - qwen3:32b", + 1925| "title": "PD - shared light tier (manual / fallback)", 1926| "provider": "ollama", - 1927| "model": "qwen3:32b", - 1928| "apiBase": "http://ROCINANTE_TAILSCALE_IP:11434" - 1929| } + 1927| "model": "qwen2.5:14b", + 1928| "apiBase": "http://PD_TAILSCALE_IP:11434" + 1929| }, + 1930| { + 1931| "title": "ROCINANTE - opportunistic heavy", + 1932| "provider": "ollama", + 1933| "model": "qwen3:32b", + 1934| "apiBase": "http://ROCINANTE_TAILSCALE_IP:11434" + 1935| } 1930| ], 1931| "tabAutocompleteModel": { 1932| "title": "N.O.M.A.D. - phi4", diff --git a/docs/architecture/ARCHITECTURE_OVERVIEW.md b/docs/architecture/ARCHITECTURE_OVERVIEW.md index eebcc82..90a51d7 100644 --- a/docs/architecture/ARCHITECTURE_OVERVIEW.md +++ b/docs/architecture/ARCHITECTURE_OVERVIEW.md @@ -40,7 +40,8 @@ Current live homelab stack plus the planned direction for the upgraded PD platfo - **Dockhand** (3230) — Docker management UI ### AI Layer -- **LiteLLM** (4000) — unified LLM proxy; routes to Ollama on PD (light) and Rocinante (heavy) +- **LiteLLM** (4000) — unified LLM proxy; PD remains the shared light tier, while Rocinante is an opportunistic heavy tier when the PC is online +- **Honcho** — memory/deriver stack on N.O.M.A.D. using local Ollama first so PD outages do not cascade into MeshMonitor/database incidents - **OpenWebUI** (8282) — chat UI with RAG, STT, and web search - **Qdrant** (6333/6334) — vector database for RAG - **Whisper** (8786 / 8787) — CPU speech-to-text on N.O.M.A.D. plus CUDA speech-to-text on Rocinante diff --git a/docs/architecture/SERVICES_DIRECTORY.md b/docs/architecture/SERVICES_DIRECTORY.md index 579f958..39de270 100644 --- a/docs/architecture/SERVICES_DIRECTORY.md +++ b/docs/architecture/SERVICES_DIRECTORY.md @@ -40,8 +40,8 @@ All homelab services, their hosts, ports, and current status. | Whisper (faster-whisper, CPU) | N.O.M.A.D. (10.5.30.7) | 8786 | ✅ Active | | Whisper (faster-whisper, CUDA large-v3) | Rocinante (10.5.30.112) | 8787 | ⚠️ Re-check from N.O.M.A.D. if Whisper-specific traffic is needed; Ollama-only recovery was validated 2026-05-27 | | SearXNG | PD | 8888 | ✅ Active | -| Ollama — light tier | PD (10.5.30.6) | 11434 | ✅ Active | -| Ollama — heavy tier | Rocinante (10.5.30.112) | 11434 | ✅ Reachable from N.O.M.A.D.; re-validated 2026-05-27 | +| Ollama — light tier | PD (10.5.30.6) | 11434 | ✅ Active; keep routine Honcho load off this host to protect MeshMonitor/shared Postgres | +| Ollama — heavy tier | Rocinante (10.5.30.112) | 11434 | ⚠️ Personal PC / opportunistic only; do not depend on it for routine background routing | | Reranker (TEI) | Serenity (10.5.30.5) | 9787 | ✅ Active | OpenClaw is no longer treated as an active PD service in the operator inventory. Only helper-artifact references remain under the school-intake planning docs. diff --git a/docs/servers/PLAUSIBLEDENABILITY.md b/docs/servers/PLAUSIBLEDENABILITY.md index eef0c1e..01ccc5c 100644 --- a/docs/servers/PLAUSIBLEDENABILITY.md +++ b/docs/servers/PLAUSIBLEDENABILITY.md @@ -47,9 +47,10 @@ Recommended models for 11GB VRAM: | `phi4` | ~8GB | Analytical tasks | Current light-tier notes: -- PD Ollama is the active light-tier endpoint for LiteLLM on `http://10.5.30.6:11434` -- Local Honcho currently points at PD Ollama over LAN (`http://10.5.30.6:11434/v1`) instead of using a separate local model host -- Current Honcho routing split keeps embeddings on `nomic-embed-text:latest`, cheap/default paths on `qwen2.5:7b`, and deeper high/max paths on `qwen2.5:14b` +- PD Ollama remains the shared light-tier endpoint for the broader AI stack on `http://10.5.30.6:11434`, but it should be protected from routine Honcho background load. +- Local Honcho now uses N.O.M.A.D.'s own Ollama on `http://127.0.0.1:11434/v1` so PD pressure does not take down MeshMonitor or shared Postgres. +- Current Honcho routing keeps embeddings on `nomic-embed-text:v1.5`, cheap/default paths on `qwen2.5:3b`, and high/max paths on `llama3.1:latest`. +- Rocinante is John's PC and should be treated as an opportunistic/manual heavy tier, not a dependable always-on backend. ## Remote Access - Pangolin VPS + Newt for external services diff --git a/docs/servers/ROCINANTE.md b/docs/servers/ROCINANTE.md index 99343d4..8d152b7 100644 --- a/docs/servers/ROCINANTE.md +++ b/docs/servers/ROCINANTE.md @@ -24,15 +24,15 @@ Ollama runs on port `11434` and hosts the large-model ("heavy") tier. | `qwen2.5:32b` | Large-context tasks | | `gemma3:27b` | Multimodal / vision | -> **LiteLLM tier:** Rocinante is the **"heavy"** tier in LiteLLM `config.yaml`. -> Heavy-tier requests from OpenWebUI and n8n route to `http://10.5.30.112:11434`. +> **LiteLLM tier:** Rocinante may serve as an **opportunistic heavy/manual tier** when the PC is online. +> Do not depend on `http://10.5.30.112:11434` for routine background routing; verify live reachability first and prefer N.O.M.A.D.-local paths for guaranteed availability. ## Current operational note -- VLAN access to Rocinante from N.O.M.A.D. was re-validated on 2026-05-27. -- Rocinante Ollama now answers from N.O.M.A.D. at `http://10.5.30.112:11434`; `/api/tags`, `/api/generate`, and OpenAI-compatible `/v1/chat/completions` all succeeded. -- Hermes configs that still pointed at the legacy `10.5.1.112` endpoint were corrected to `10.5.30.112`. -- Re-check CUDA Whisper on `http://10.5.30.112:8787/health` separately if Whisper-specific reachability is needed; this pass validated Ollama end-to-end. +- Rocinante is John's personal PC and is not a dependable always-on homelab tier. +- Treat Ollama/Whisper on Rocinante as opportunistic capacity when the machine is online, not as a required background dependency. +- If N.O.M.A.D. or PD need guaranteed inference, prefer local N.O.M.A.D. routing first and keep PD protected from routine background load. +- Re-check live reachability from N.O.M.A.D. before any Rocinante-dependent maintenance or cutover. ## Notes diff --git a/n8n-workflows/README.md b/n8n-workflows/README.md index 613c8de..95d35da 100644 --- a/n8n-workflows/README.md +++ b/n8n-workflows/README.md @@ -171,10 +171,16 @@ curl -X PUT http://10.5.30.6:6333/collections/knowledge_base \ (768 dimensions matches `nomic-embed-text`. If you use a different embedding model, adjust the size.) -### Pull nomic-embed-text into Ollama (required for workflows 02, 05, 08) +### Pull `nomic-embed-text` into the Ollama host backing these workflows (required for workflows 02, 05, 08) +For N.O.M.A.D.-local workflows / Honcho-style routing: ```bash -curl http://10.5.30.6:11434/api/pull -d '{"name": "nomic-embed-text"}' +curl http://127.0.0.1:11434/api/pull -d '{"name": "nomic-embed-text:v1.5"}' +``` + +For PD-hosted shared workflows, use the PD endpoint explicitly instead: +```bash +curl http://10.5.30.6:11434/api/pull -d '{"name": "nomic-embed-text:latest"}' ``` ### Paperless Webhook (required for workflows 03, 05, 17, and 19)