Rebalance homelab AI routing around NOMAD-first inference

This commit is contained in:
Fizzlepoof
2026-06-13 23:54:56 +00:00
parent e19d4a5be3
commit 43b94813ca
7 changed files with 40 additions and 25 deletions

View File

@@ -575,7 +575,7 @@
575|Trigger: Cron node (every Sunday at 10:00 AM)
576|Step 1: Postgres node → SELECT watch history for past 7 days
577|Step 2: Format data as structured text
578|Step 3: HTTP Request → Ollama API (POST to PD's qwen2.5:14b)
578|Step 3: HTTP Request → Ollama API (POST to the local/default automation model host, preferably N.O.M.A.D.-local rather than PD)
579| - Prompt: "Summarize this week's viewing in a fun digest: {data}"
580|Step 4: HTTP Request → Gotify (send digest)
581|```
@@ -1093,9 +1093,9 @@
1388|
1389|LiteLLM sits in front of your three Ollama instances and presents a single OpenAI-compatible API at `http://litellm:4000`. It handles:
1390|
1391|- **Model routing** — requests for `qwen3:32b` go to ROCINANTE, `qwen2.5:14b` goes to PD, `phi4` goes to N.O.M.A.D.
1392|- **Failover** — if ROCINANTE is offline, LiteLLM can fall back to PD automatically
1393|- **Load balancing** — distribute requests across instances running the same model
- **Model routing** — current reliable default is N.O.M.A.D.-local Ollama for background/automation paths, with PD kept available for shared light-tier use and Rocinante treated as optional/manual heavy capacity.
- **Failover** — avoid designs that require Rocinante to be online; degrade to N.O.M.A.D.-local or explicitly operator-invoked PD paths instead of automatic dependence on a personal PC.
- **Load balancing** — distribute requests across instances running the same model when you have multiple dependable backends, but do not assume Rocinante qualifies as one.
1394|- **Usage tracking** — logs token counts, latency, and costs per model/user via its built-in database
1395|
1396|### 7.2 — Scaffold Directories
@@ -1621,17 +1621,23 @@
1916|{
1917| "models": [
1918| {
1919| "title": "PD - qwen2.5:14b",
1919| "title": "N.O.M.A.D. - qwen2.5:3b",
1920| "provider": "ollama",
1921| "model": "qwen2.5:14b",
1922| "apiBase": "http://PD_TAILSCALE_IP:11434"
1921| "model": "qwen2.5:3b",
1922| "apiBase": "http://NOMAD_LOCAL_OR_TAILSCALE_IP:11434"
1923| },
1924| {
1925| "title": "ROCINANTE - qwen3:32b",
1925| "title": "PD - shared light tier (manual / fallback)",
1926| "provider": "ollama",
1927| "model": "qwen3:32b",
1928| "apiBase": "http://ROCINANTE_TAILSCALE_IP:11434"
1929| }
1927| "model": "qwen2.5:14b",
1928| "apiBase": "http://PD_TAILSCALE_IP:11434"
1929| },
1930| {
1931| "title": "ROCINANTE - opportunistic heavy",
1932| "provider": "ollama",
1933| "model": "qwen3:32b",
1934| "apiBase": "http://ROCINANTE_TAILSCALE_IP:11434"
1935| }
1930| ],
1931| "tabAutocompleteModel": {
1932| "title": "N.O.M.A.D. - phi4",