fizzlepoof/truenas-stacks
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Rebalance homelab AI routing around NOMAD-first inference

Browse Source
This commit is contained in:
Fizzlepoof
2026-06-13 23:54:56 +00:00
parent e19d4a5be3
commit 43b94813ca
7 changed files with 40 additions and 25 deletions
Download Patch File Download Diff File Expand all files Collapse all files

7
docs/servers/PLAUSIBLEDENABILITY.md
View File

@@ -47,9 +47,10 @@ Recommended models for 11GB VRAM:
| `phi4` | ~8GB | Analytical tasks |
Current light-tier notes:
- PD Ollama is the active light-tier endpoint for LiteLLM on `http://10.5.30.6:11434`
- Local Honcho currently points at PD Ollama over LAN (`http://10.5.30.6:11434/v1`) instead of using a separate local model host
- Current Honcho routing split keeps embeddings on `nomic-embed-text:latest`, cheap/default paths on `qwen2.5:7b`, and deeper high/max paths on `qwen2.5:14b`
- PD Ollama remains the shared light-tier endpoint for the broader AI stack on `http://10.5.30.6:11434`, but it should be protected from routine Honcho background load.
- Local Honcho now uses N.O.M.A.D.'s own Ollama on `http://127.0.0.1:11434/v1` so PD pressure does not take down MeshMonitor or shared Postgres.
- Current Honcho routing keeps embeddings on `nomic-embed-text:v1.5`, cheap/default paths on `qwen2.5:3b`, and high/max paths on `llama3.1:latest`.
- Rocinante is John's PC and should be treated as an opportunistic/manual heavy tier, not a dependable always-on backend.
## Remote Access
- Pangolin VPS + Newt for external services