Rebalance homelab AI routing around NOMAD-first inference
This commit is contained in:
@@ -575,7 +575,7 @@
|
||||
575|Trigger: Cron node (every Sunday at 10:00 AM)
|
||||
576|Step 1: Postgres node → SELECT watch history for past 7 days
|
||||
577|Step 2: Format data as structured text
|
||||
578|Step 3: HTTP Request → Ollama API (POST to PD's qwen2.5:14b)
|
||||
578|Step 3: HTTP Request → Ollama API (POST to the local/default automation model host, preferably N.O.M.A.D.-local rather than PD)
|
||||
579| - Prompt: "Summarize this week's viewing in a fun digest: {data}"
|
||||
580|Step 4: HTTP Request → Gotify (send digest)
|
||||
581|```
|
||||
@@ -1093,9 +1093,9 @@
|
||||
1388|
|
||||
1389|LiteLLM sits in front of your three Ollama instances and presents a single OpenAI-compatible API at `http://litellm:4000`. It handles:
|
||||
1390|
|
||||
1391|- **Model routing** — requests for `qwen3:32b` go to ROCINANTE, `qwen2.5:14b` goes to PD, `phi4` goes to N.O.M.A.D.
|
||||
1392|- **Failover** — if ROCINANTE is offline, LiteLLM can fall back to PD automatically
|
||||
1393|- **Load balancing** — distribute requests across instances running the same model
|
||||
- **Model routing** — current reliable default is N.O.M.A.D.-local Ollama for background/automation paths, with PD kept available for shared light-tier use and Rocinante treated as optional/manual heavy capacity.
|
||||
- **Failover** — avoid designs that require Rocinante to be online; degrade to N.O.M.A.D.-local or explicitly operator-invoked PD paths instead of automatic dependence on a personal PC.
|
||||
- **Load balancing** — distribute requests across instances running the same model when you have multiple dependable backends, but do not assume Rocinante qualifies as one.
|
||||
1394|- **Usage tracking** — logs token counts, latency, and costs per model/user via its built-in database
|
||||
1395|
|
||||
1396|### 7.2 — Scaffold Directories
|
||||
@@ -1621,17 +1621,23 @@
|
||||
1916|{
|
||||
1917| "models": [
|
||||
1918| {
|
||||
1919| "title": "PD - qwen2.5:14b",
|
||||
1919| "title": "N.O.M.A.D. - qwen2.5:3b",
|
||||
1920| "provider": "ollama",
|
||||
1921| "model": "qwen2.5:14b",
|
||||
1922| "apiBase": "http://PD_TAILSCALE_IP:11434"
|
||||
1921| "model": "qwen2.5:3b",
|
||||
1922| "apiBase": "http://NOMAD_LOCAL_OR_TAILSCALE_IP:11434"
|
||||
1923| },
|
||||
1924| {
|
||||
1925| "title": "ROCINANTE - qwen3:32b",
|
||||
1925| "title": "PD - shared light tier (manual / fallback)",
|
||||
1926| "provider": "ollama",
|
||||
1927| "model": "qwen3:32b",
|
||||
1928| "apiBase": "http://ROCINANTE_TAILSCALE_IP:11434"
|
||||
1929| }
|
||||
1927| "model": "qwen2.5:14b",
|
||||
1928| "apiBase": "http://PD_TAILSCALE_IP:11434"
|
||||
1929| },
|
||||
1930| {
|
||||
1931| "title": "ROCINANTE - opportunistic heavy",
|
||||
1932| "provider": "ollama",
|
||||
1933| "model": "qwen3:32b",
|
||||
1934| "apiBase": "http://ROCINANTE_TAILSCALE_IP:11434"
|
||||
1935| }
|
||||
1930| ],
|
||||
1931| "tabAutocompleteModel": {
|
||||
1932| "title": "N.O.M.A.D. - phi4",
|
||||
|
||||
Reference in New Issue
Block a user