fizzlepoof/truenas-stacks
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Rebalance homelab AI routing around NOMAD-first inference

Browse Source
This commit is contained in:
Fizzlepoof
2026-06-13 23:54:56 +00:00
parent e19d4a5be3
commit 43b94813ca
7 changed files with 40 additions and 25 deletions
Download Patch File Download Diff File Expand all files Collapse all files

7
docs/servers/PLAUSIBLEDENABILITY.md
View File

@@ -47,9 +47,10 @@ Recommended models for 11GB VRAM:
| `phi4` | ~8GB | Analytical tasks |
Current light-tier notes:
- PD Ollama is the active light-tier endpoint for LiteLLM on `http://10.5.30.6:11434`
- Local Honcho currently points at PD Ollama over LAN (`http://10.5.30.6:11434/v1`) instead of using a separate local model host
- Current Honcho routing split keeps embeddings on `nomic-embed-text:latest`, cheap/default paths on `qwen2.5:7b`, and deeper high/max paths on `qwen2.5:14b`
- PD Ollama remains the shared light-tier endpoint for the broader AI stack on `http://10.5.30.6:11434`, but it should be protected from routine Honcho background load.
- Local Honcho now uses N.O.M.A.D.'s own Ollama on `http://127.0.0.1:11434/v1` so PD pressure does not take down MeshMonitor or shared Postgres.
- Current Honcho routing keeps embeddings on `nomic-embed-text:v1.5`, cheap/default paths on `qwen2.5:3b`, and high/max paths on `llama3.1:latest`.
- Rocinante is John's PC and should be treated as an opportunistic/manual heavy tier, not a dependable always-on backend.
## Remote Access
- Pangolin VPS + Newt for external services

12
docs/servers/ROCINANTE.md
View File

@@ -24,15 +24,15 @@ Ollama runs on port `11434` and hosts the large-model ("heavy") tier.
| `qwen2.5:32b` | Large-context tasks |
| `gemma3:27b` | Multimodal / vision |
> **LiteLLM tier:** Rocinante is the **"heavy"** tier in LiteLLM `config.yaml`.
> Heavy-tier requests from OpenWebUI and n8n route to `http://10.5.30.112:11434`.
> **LiteLLM tier:** Rocinante may serve as an **opportunistic heavy/manual tier** when the PC is online.
> Do not depend on `http://10.5.30.112:11434` for routine background routing; verify live reachability first and prefer N.O.M.A.D.-local paths for guaranteed availability.
## Current operational note
- VLAN access to Rocinante from N.O.M.A.D. was re-validated on 2026-05-27.
- Rocinante Ollama now answers from N.O.M.A.D. at `http://10.5.30.112:11434`; `/api/tags`, `/api/generate`, and OpenAI-compatible `/v1/chat/completions` all succeeded.
- Hermes configs that still pointed at the legacy `10.5.1.112` endpoint were corrected to `10.5.30.112`.
- Re-check CUDA Whisper on `http://10.5.30.112:8787/health` separately if Whisper-specific reachability is needed; this pass validated Ollama end-to-end.
- Rocinante is John's personal PC and is not a dependable always-on homelab tier.
- Treat Ollama/Whisper on Rocinante as opportunistic capacity when the machine is online, not as a required background dependency.
- If N.O.M.A.D. or PD need guaranteed inference, prefer local N.O.M.A.D. routing first and keep PD protected from routine background load.
- Re-check live reachability from N.O.M.A.D. before any Rocinante-dependent maintenance or cutover.
## Notes