truenas-stacks/HOMELAB_BUILDOUT_PLAN.md at 1eb4bb9981ef154f20d78a3cac5c2871a34ac2e2

Files

Fizzlepoof 43b94813ca Rebalance homelab AI routing around NOMAD-first inference

2026-06-13 23:54:56 +00:00

66 KiB

Raw Blame History

 1|# Homelab Expansion — Full Buildout Plan
 2|
 3|> Generated: 2026-05-04
 4|> Based on: HOMELAB_EXPANSION_PLAN.md, ARCHITECTURE_OVERVIEW.md, STACK_STANDARDS.md
 5|
 6|This document contains everything needed to deploy all six expansion phases — docker-compose files, .env templates, directory scaffolding commands, database init SQL, Pangolin configuration, and validation steps. Each phase follows the established stack standards and deployment checklist.
 7|
 8|---
 9|
10|## Table of Contents
11|
12|1. [Prerequisites & Conventions](#prerequisites--conventions)
13|2. [Phase 1 — Gotify (Notifications)](#phase-1--gotify-notifications)
14|3. [Phase 2 — Qdrant (Vector Database)](#phase-2--qdrant-vector-database)
15|4. [Phase 3 — n8n (Workflow Automation)](#phase-3--n8n-workflow-automation)
16|5. [Phase 4 — Paperless-NGX (Document Intelligence)](#phase-4--paperless-ngx-document-intelligence)
17|6. [Phase 5 — Home Assistant (Home Automation)](#phase-5--home-assistant-home-automation)
18|7. [Phase 6 — Grafana + Prometheus (Observability)](#phase-6--grafana--prometheus-observability)
19|8. [Phase 7 — LiteLLM (AI Gateway)](#phase-7--litellm-ai-gateway)
20|9. [Phase 8 — Reranker (RAG Quality)](#phase-8--reranker-rag-quality)
21|10. [Phase 9 — faster-whisper (Speech-to-Text)](#phase-9--faster-whisper-speech-to-text)
22|11. [Additional Tools Setup](#additional-tools-setup)
23|12. [Deployment Order Summary](#deployment-order-summary)
24|13. [Post-Deployment Validation Master Checklist](#post-deployment-validation-master-checklist)
25|
26|---
27|
28|## Prerequisites & Conventions
29|
30|### Deployment Standards (recap)
31|
32|All stacks on PlausibleDeniability follow the same pattern:
33|
34|- **Repo root:** `/mnt/docker-ssd/docker/compose`
35|- **Validate before deploy:** `docker compose --env-file .env config`
36|- **Deploy:** `docker compose --env-file .env up -d`
37|- **Teardown:** `docker compose --env-file .env down`
38|
39|### Storage Tiers
40|
41|| Tier | Mount | Use For |
42||------|-------|---------|
43|| SSD | `/mnt/docker-ssd/docker/appdata/<service>` | Write-heavy, SQLite, GPU/model, databases |
44|| Tank | `/mnt/tank/docker/appdata/<service>` | General appdata, configs, uploads |
45|| Unraid | `/mnt/unraid/data/media/` | Media libraries only |
46|
47|### Networks
48|
49|| Network | Created By | Purpose |
50||---------|-----------|---------|
51|| `ix-databases_shared-databases` | databases stack | Access to shared-postgres, shared-mariadb, shared-redis |
52|| `pangolin` | newt (infrastructure stack) | Reverse proxy / external exposure |
53|
54|### Secret Generation Commands
55|
56|```bash
57|# Database passwords
58|openssl rand -hex 24
59|
60|# JWT / encryption keys
61|openssl rand -hex 32
62|
63|# Paperless secret key
64|python3 -c "import secrets; print(secrets.token_urlsafe(50))"
65|```
66|
67|### Deployment Order
68|
69|```
70|databases (already running) → infrastructure (already running)
71|  → Phase 1: Gotify
72|  → Phase 2: Qdrant  (creates the ai-services network)
73|  → Phase 3: n8n     (depends on Gotify + Qdrant — joins ai-services as external)
74|  → Phase 4: Paperless-NGX (depends on n8n for automation hooks)
75|  → Phase 5: Home Assistant (depends on n8n for heavy automation)
76|  → Phase 6: Grafana + Prometheus (on N.O.M.A.D., independent but benefits from all above)
77|```
78|
79|> **Important:** The `ai` stack (Phase 2) must be deployed before the `automation` stack (Phase 3) because n8n declares `ai-services` as an external network. If the `ai` stack isn't up, `docker compose up` for `automation` will fail with a missing network error.
80|
81|---
82|
83|## Phase 1 — Gotify (Notifications)
84|
85|**Host:** PlausibleDeniability
86|**Stack directory:** `/mnt/docker-ssd/docker/compose/automation/`
87|**Why first:** Every subsequent phase sends notifications through Gotify. It's the output bus.
88|
89|### 1.1 — Scaffold Directories
90|
91|```bash
92|# Appdata on SSD (SQLite backend — must not be on NFS)
93|sudo mkdir -p /mnt/docker-ssd/docker/appdata/gotify
94|
95|# Stack directory (may already exist if automation/ is planned)
96|sudo mkdir -p /mnt/docker-ssd/docker/compose/automation
97|```
98|
99|### 1.2 — Database Init

100| 101|None required — Gotify uses an embedded SQLite database stored in its data volume. 102| 103|### 1.3 — docker-compose.yaml 104| 105|Add to /mnt/docker-ssd/docker/compose/automation/docker-compose.yaml: 106| 107|yaml 108|name: automation 109| 110|services: 111| gotify: 112| image: gotify/server:2.6.1 113| container_name: gotify 114| restart: unless-stopped 115| ports: 116| - "8484:80" 117| environment: 118| TZ: ${TZ} 119| GOTIFY_DEFAULTUSER_PASS: ${GOTIFY_ADMIN_PASS} 120| GOTIFY_SERVER_PORT: 80 121| GOTIFY_DATABASE_DIALECT: sqlite3 122| GOTIFY_DATABASE_CONNECTION: data/gotify.db 123| volumes: 124| - /mnt/docker-ssd/docker/appdata/gotify:/app/data 125| networks: 126| - pangolin 127| - default 128| 129|networks: 130| pangolin: 131| external: true 132| 133| 134|> Image note: gotify/server:2.6.1 is the latest stable as of May 2026. Check Docker Hub before deploying — pin to the exact version. 135| 136|### 1.4 — .env.example 137| 138|bash 139|# /mnt/docker-ssd/docker/compose/automation/.env.example 140| 141|TZ=America/New_York 142|GOTIFY_ADMIN_PASS=CHANGE_ME 143| 144| 145|### 1.5 — .env (create from example) 146| 147|bash 148|cd /mnt/docker-ssd/docker/compose/automation 149|cp .env.example .env 150|# Edit .env with real values: 151|# GOTIFY_ADMIN_PASS=$(openssl rand -hex 16) 152|nano .env 153| 154| 155|### 1.6 — Pangolin Configuration 156| 157|In your Pangolin dashboard, create a new resource: 158| 159|| Field | Value | 160||-------|-------| 161|| Domain | gotify.paccoco.com | 162|| Scheme | http | 163|| Host | gotify | 164|| Port | 80 | 165|| Network | Docker service name resolution via pangolin network | 166| 167|### 1.7 — Validate & Deploy 168| 169|bash 170|cd /mnt/docker-ssd/docker/compose/automation 171|docker compose --env-file .env config 172|docker compose --env-file .env up -d 173| 174| 175|### 1.8 — Post-Deploy Verification 176| 177|bash 178|# Container running? 179|docker ps --filter name=gotify 180| 181|# Clean startup logs? 182|docker logs gotify --tail 20 183| 184|# Mounts correct? 185|docker inspect gotify --format '{{json .Mounts}}' | python3 -m json.tool 186| 187|# Quick health check 188|curl -s http://localhost:8484/health 189| 190|# Test notification via API 191|curl -s "http://localhost:8484/message?token=*** \ 192| -F "title=Homelab" \ 193| -F "message=Gotify is online" \ 194| -F "priority=5" 195| 196| 197|### 1.9 — First-Run Setup 198| 199|1. Navigate to https://gotify.paccoco.com 200|2. Log in with the admin password from .env 201|3. Change the default admin password in the UI 202|4. Create application tokens for each notification source: 203| - n8n-workflows — for all n8n automations 204| - infrastructure-alerts — for Uptime Kuma, Grafana, etc. 205| - media-notifications — for Sonarr/Radarr/Tautulli hooks 206| - home-assistant — for HA automations 207|5. Create client tokens for each receiving device (phone, desktop) 208|6. Install the Gotify Android app and configure with your client token 209| 210|--- 211| 212|## Phase 2 — AI Stack (Vector DB, LLM, Embeddings, Reranker, STT) 213| 214|Host: PlausibleDeniability 215|Stack directory: /mnt/docker-ssd/docker/compose/ai/ 216|Status: DEPLOYED AND VERIFIED (2026-05-05) 217| 218|> DEPLOYED NOTE: This phase was deployed as a complete AI stack with 6 services: Ollama (11434), OpenWebUI (8282), Qdrant (6333/6334), LiteLLM (4000), Reranker/TEI (8787), and faster-whisper (8786). The compose below shows the original Qdrant-only plan — see the actual running compose on PD at /mnt/docker-ssd/docker/compose/ai/docker-compose.yaml and the project_ai_stack_deployed.md memory for port mappings and fixes. Key lesson: always use 10.5.30.6 not localhost for health checks on TrueNAS Scale. 219| 220|### 2.1 — Scaffold Directories 221| 222|bash 223|# Appdata on SSD (write-heavy vector storage) 224|sudo mkdir -p /mnt/docker-ssd/docker/appdata/qdrant/storage 225|sudo mkdir -p /mnt/docker-ssd/docker/appdata/qdrant/snapshots 226| 227|# Stack directory 228|sudo mkdir -p /mnt/docker-ssd/docker/compose/ai 229| 230| 231|### 2.2 — docker-compose.yaml 232| 233|Create or update /mnt/docker-ssd/docker/compose/ai/docker-compose.yaml: 234| 235|yaml 236|name: ai 237| 238|services: 239| qdrant: 240| image: qdrant/qdrant:v1.14.0 241| container_name: qdrant 242| restart: unless-stopped 243| ports: 244| - "6333:6333" # REST API 245| - "6334:6334" # gRPC 246| environment: 247| TZ: ${TZ} 248| QDRANT__SERVICE__GRPC_PORT: 6334 249| QDRANT__STORAGE__STORAGE_PATH: /qdrant/storage 250| QDRANT__STORAGE__SNAPSHOTS_PATH: /qdrant/snapshots 251| volumes: 252| - /mnt/docker-ssd/docker/appdata/qdrant/storage:/qdrant/storage 253| - /mnt/docker-ssd/docker/appdata/qdrant/snapshots:/qdrant/snapshots 254| networks: 255| - ai-services 256| - default 257| 258| # ------------------------------------------------------- 259| # Ollama and OpenWebUI go here when deployed. 260| # They share this stack and the default + ai-services networks so 261| # OpenWebUI can reach Qdrant at http://qdrant:6333 262| # ------------------------------------------------------- 263| 264| # ollama: 265| # image: ollama/ollama:latest 266| # container_name: ollama 267| # ... 268| 269| # openwebui: 270| # image: ghcr.io/open-webui/open-webui:main 271| # container_name: openwebui 272| # ... 273| 274|networks: 275| ai-services: 276| name: ai-services 277| 278| 279|> Image note: qdrant/qdrant:v1.14.0 is the latest stable as of May 2026. Check GitHub releases before deploying. 280| 281|> Cross-stack connectivity: The ai-services network is defined here with an explicit name: so other stacks (like automation/n8n) can declare it as external: true and reach Qdrant by service name (http://qdrant:6333). This follows the same pattern as ix-databases_shared-databases. 282| 283|### 2.3 — .env.example 284| 285|bash 286|# /mnt/docker-ssd/docker/compose/ai/.env.example 287| 288|TZ=America/New_York 289| 290| 291|### 2.4 — Validate & Deploy 292| 293|bash 294|cd /mnt/docker-ssd/docker/compose/ai 295|cp .env.example .env 296|nano .env # Set timezone 297| 298|docker compose --env-file .env config 299|docker compose --env-file .env up -d 300| 301| 302|### 2.5 — Post-Deploy Verification 303| 304|bash 305|# Container running? 306|docker ps --filter name=qdrant 307| 308|# Clean startup? 309|docker logs qdrant --tail 20 310| 311|# REST API responding? 312|curl -s http://localhost:6333/healthz 313|# Expected: {"title":"qdrant - vectorass engine","version":"1.14.0","commit":"..."} 314| 315|# Create a test collection 316|curl -X PUT http://localhost:6333/collections/test_collection \ 317| -H "Content-Type: application/json" \ 318| -d '{ 319| "vectors": { 320| "size": 384, 321| "distance": "Cosine" 322| } 323| }' 324| 325|# Verify it exists 326|curl -s http://localhost:6333/collections | python3 -m json.tool 327| 328|# Clean up test 329|curl -X DELETE http://localhost:6333/collections/test_collection 330| 331| 332|### 2.6 — OpenWebUI Integration (when deployed) 333| 334|When Ollama and OpenWebUI are brought online in this same stack, configure OpenWebUI's RAG settings: 335| 336|1. Go to OpenWebUI → Admin → Settings → Documents 337|2. Set the vector database to Qdrant 338|3. Endpoint: http://qdrant:6333 (Docker DNS within the ai stack network) 339|4. Collection name: openwebui_docs (or your preference) 340| 341|### 2.7 — Collections to Create (for n8n in Phase 3) 342| 343|These collections will be created programmatically by n8n workflows, but for reference: 344| 345|| Collection | Vector Size | Content | 346||-----------|-------------|---------| 347|| homelab_docs | 384 (nomic-embed-text) | Homelab markdown documentation | 348|| gitea_commits | 384 | Gitea commit messages + diffs | 349|| media_metadata | 384 | Plex/Tautulli metadata | 350|| obsidian_notes | 384 | Personal notes from Obsidian vault | 351| 352|> Vector size note: 384 is the dimension for nomic-embed-text via Ollama. If you use a different embedding model, adjust accordingly. 353| 354|--- 355| 356|## Phase 3 — n8n (Workflow Automation) 357| 358|Host: PlausibleDeniability 359|Stack directory: /mnt/docker-ssd/docker/compose/automation/ (same stack as Gotify) 360|Why third: n8n is the orchestration backbone — it ties Gotify, Qdrant, Ollama, and all triggers together. 361| 362|### 3.1 — Scaffold Directories 363| 364|bash 365|# Config on tank (workflow definitions, credentials store) 366|# NOTE: n8n also writes execution logs here. If execution logging becomes 367|# heavy (many workflows running frequently), consider moving to SSD. 368|# For typical homelab usage (~20 workflows), tank is fine. 369|sudo mkdir -p /mnt/tank/docker/appdata/n8n 370| 371|# Set ownership — n8n runs as UID 1000 (node user) 372|sudo chown 1000:1000 /mnt/tank/docker/appdata/n8n 373| 374| 375|### 3.2 — Database Init 376| 377|n8n uses the existing shared-postgres. Create its database and user: 378| 379|bash 380|docker exec -i shared-postgres psql -U postgres <<'SQL' 381|CREATE USER n8n WITH PASSWORD 'REPLACE_WITH_GENERATED_PASSWORD'; 382|CREATE DATABASE n8n OWNER n8n; 383|GRANT ALL PRIVILEGES ON DATABASE n8n TO n8n; 384|SQL 385| 386| 387|Generate the password first: 388|bash 389|openssl rand -hex 24 390| 391| 392|### 3.3 — docker-compose.yaml 393| 394|Update /mnt/docker-ssd/docker/compose/automation/docker-compose.yaml to add n8n alongside Gotify: 395| 396|yaml 397|name: automation 398| 399|services: 400| gotify: 401| image: gotify/server:2.6.1 402| container_name: gotify 403| restart: unless-stopped 404| ports: 405| - "8484:80" 406| environment: 407| TZ: ${TZ} 408| GOTIFY_DEFAULTUSER_PASS: ${GOTIFY_ADMIN_PASS} 409| GOTIFY_SERVER_PORT: 80 410| GOTIFY_DATABASE_DIALECT: sqlite3 411| GOTIFY_DATABASE_CONNECTION: data/gotify.db 412| volumes: 413| - /mnt/docker-ssd/docker/appdata/gotify:/app/data 414| networks: 415| - pangolin 416| - default 417| 418| n8n: 419| image: docker.n8n.io/n8nio/n8n:1.88.0 420| container_name: n8n 421| restart: unless-stopped 422| ports: 423| - "5678:5678" 424| environment: 425| TZ: ${TZ} 426| # Database 427| DB_TYPE: postgresdb 428| DB_POSTGRESDB_HOST: shared-postgres 429| DB_POSTGRESDB_PORT: 5432 430| DB_POSTGRESDB_DATABASE: ${N8N_DB_NAME} 431| DB_POSTGRESDB_USER: ${N8N_DB_USER} 432| DB_POSTGRESDB_PASSWORD: ${N8N_DB_PASS} 433| # Security 434| N8N_ENCRYPTION_KEY: ${N8N_ENCRYPTION_KEY} 435| # Webhook / Reverse Proxy 436| N8N_HOST: ${N8N_HOST} 437| N8N_PROTOCOL: https 438| N8N_PORT: 5678 439| WEBHOOK_URL: https://${N8N_HOST}/ 440| N8N_PROXY_HOPS: 1 441| # General 442| GENERIC_TIMEZONE: ${TZ} 443| N8N_DIAGNOSTICS_ENABLED: false 444| N8N_PERSONALIZATION_ENABLED: false 445| volumes: 446| - /mnt/tank/docker/appdata/n8n:/home/node/.n8n 447| networks: 448| - pangolin 449| - ix-databases_shared-databases 450| - ai-services 451| - default 452| depends_on: 453| - gotify 454| 455|networks: 456| pangolin: 457| external: true 458| ix-databases_shared-databases: 459| external: true 460| ai-services: 461| external: true 462| 463| 464|> Image note: docker.n8n.io/n8nio/n8n:1.88.0 — n8n uses calendar-ish versioning. Check n8n releases for the latest stable. The docker.n8n.io registry is preferred for production. 465| 466|> Cross-stack access: n8n joins ai-services (created by the ai stack) so it can reach Qdrant at http://qdrant:6333 by Docker DNS. It also joins ix-databases_shared-databases for Postgres access — same pattern. 467| 468|### 3.4 — .env.example (updated for both services) 469| 470|bash 471|# /mnt/docker-ssd/docker/compose/automation/.env.example 472| 473|TZ=America/New_York 474| 475|# Gotify 476|GOTIFY_ADMIN_PASS=CHANGE_ME 477| 478|# n8n — Database 479|N8N_DB_NAME=n8n 480|N8N_DB_USER=n8n 481|N8N_DB_PASS=CHANGE_ME 482| 483|# n8n — Security 484|N8N_ENCRYPTION_KEY=CHANGE_ME 485| 486|# n8n — Hostname 487|N8N_HOST=n8n.paccoco.com 488| 489| 490|### 3.5 — Pangolin Configuration 491| 492|| Field | Value | 493||-------|-------| 494|| Domain | n8n.paccoco.com | 495|| Scheme | http | 496|| Host | n8n | 497|| Port | 5678 | 498|| Headers | Forward X-Forwarded-Proto: https | 499| 500|### 3.6 — Validate & Deploy 501| 502|bash 503|cd /mnt/docker-ssd/docker/compose/automation 504| 505|# Update .env with real passwords 506|nano .env 507| 508|docker compose --env-file .env config 509|docker compose --env-file .env up -d 510| 511| 512|### 3.7 — Post-Deploy Verification 513| 514|bash 515|# Both containers running? 516|docker ps --filter name=gotify --filter name=n8n 517| 518|# n8n logs clean? 519|docker logs n8n --tail 30 520| 521|# n8n on correct networks? 522|docker inspect n8n --format '{{json .NetworkSettings.Networks}}' | python3 -m json.tool 523| 524|# n8n UI accessible? 525|curl -s -o /dev/null -w "%{http_code}" http://localhost:5678/ 526|# Expected: 200 527| 528|# Postgres connection working? (check logs for DB migration messages) 529|docker logs n8n 2>&1 | grep -i "migrat" 530| 531| 532|### 3.8 — First-Run Setup 533| 534|1. Navigate to https://n8n.paccoco.com 535|2. Create your admin account 536|3. Install community nodes you'll need: 537| - n8n-nodes-gotify (if available) or use HTTP Request node 538| - Ollama nodes (built-in as of n8n 1.x) 539| 540|### 3.9 — Workflow Blueprints 541| 542|Below are starter workflow descriptions for each planned automation. These are meant to be built in the n8n UI — the structure and node types are described so you can wire them up. 543| 544|#### Media Pipeline Workflows 545| 546|Workflow: Sonarr/Radarr Download Notification 547| 548|Trigger: Webhook node (POST from Sonarr/Radarr on download/import) 549|Step 1: Extract series/movie name, quality, file path from webhook body 550|Step 2: HTTP Request → TMDB API to fetch poster image URL 551|Step 3: HTTP Request → Gotify REST API (POST /message) 552| - Title: "New Download: {title}" 553| - Message: "{quality} — {episodeTitle or year}" 554| - Priority: 5 555| - Extras: attach poster URL as markdown image 556|Also: HTTP Request → Discord webhook (formatted embed with poster) 557| 558| 559|Configure Sonarr/Radarr webhooks: 560|- Sonarr: Settings → Connect → Webhook → URL: https://n8n.paccoco.com/webhook/sonarr 561|- Radarr: Settings → Connect → Webhook → URL: https://n8n.paccoco.com/webhook/radarr 562| 563|Workflow: Tautulli Play Logging 564| 565|Trigger: Webhook node (POST from Tautulli on playback start) 566|Step 1: Extract user, media title, player, quality from payload 567|Step 2: Postgres node → INSERT into watch_history table 568|Step 3: (Optional) Gotify notification for specific users/media 569| 570| 571|Tautulli config: Settings → Notification Agents → Webhook → URL: https://n8n.paccoco.com/webhook/tautulli-play 572| 573|Workflow: Weekly Watch Digest 574| 575|Trigger: Cron node (every Sunday at 10:00 AM) 576|Step 1: Postgres node → SELECT watch history for past 7 days 577|Step 2: Format data as structured text 578|Step 3: HTTP Request → Ollama API (POST to the local/default automation model host, preferably N.O.M.A.D.-local rather than PD) 579| - Prompt: "Summarize this week's viewing in a fun digest: {data}" 580|Step 4: HTTP Request → Gotify (send digest) 581| 582| 583|#### Infrastructure Monitoring Workflows 584| 585|Workflow: Uptime Kuma Enhanced Alerts 586| 587|Trigger: Webhook node (from Uptime Kuma notification) 588|Step 1: Extract monitor name, status, response time 589|Step 2: HTTP Request → Netdata API for related metrics context 590|Step 3: HTTP Request → Gotify 591| - Title: "🔴 {monitor} DOWN" or "🟢 {monitor} UP" 592| - Message: include Netdata context (CPU, mem, disk at time of alert) 593| - Priority: 8 (high for down, 3 for recovery) 594| 595| 596|Workflow: ZFS Pool Utilization Alert 597| 598|Trigger: Cron node (every 6 hours) 599|Step 1: SSH node → Serenity: `zpool list -Hp malcolm` 600|Step 2: Parse capacity percentage 601|Step 3: IF capacity > 85% → Gotify alert (priority 8) 602|Step 4: IF capacity > 90% → Gotify alert (priority 10) + Discord webhook 603| 604| 605|Workflow: Grafana Alert Remediation 606| 607|Trigger: Webhook node (from Grafana alerting) 608|Step 1: Parse alert labels (container, host, metric) 609|Step 2: Switch node → route by alert type: 610| - High CPU container → SSH → docker restart {container} 611| - Disk full → SSH → pause qBittorrent, notify via Gotify 612| - Memory pressure → Gotify alert only (manual intervention) 613|Step 3: Log action taken to Postgres 614| 615| 616|#### Homelab Ops Workflows 617| 618|Workflow: Gitea Commit Summary 619| 620|Trigger: Webhook node (Gitea webhook on push to truenas-stacks) 621|Step 1: Extract commit messages, author, files changed 622|Step 2: HTTP Request → Ollama API 623| - Prompt: "Summarize this commit in one sentence: {commit_message}" 624|Step 3: HTTP Request → Gotify 625| - Title: "Commit to truenas-stacks" 626| - Message: Ollama-generated summary 627| 628| 629|Gitea config: Repository → Settings → Webhooks → URL: https://n8n.paccoco.com/webhook/gitea-push 630| 631|Workflow: qBittorrent Auto-Rescan 632| 633|Trigger: Webhook or polling (qBittorrent API for completed+moved torrents) 634|Step 1: Determine if file is in Sonarr or Radarr path 635|Step 2: HTTP Request → Sonarr API (POST /command → RescanSeries) 636| OR HTTP Request → Radarr API (POST /command → RescanMovie) 637|Step 3: Gotify notification confirming rescan triggered 638| 639| 640|#### AI Pipeline Workflows 641| 642|Workflow: URL Digest Pipeline 643| 644|Trigger: Webhook node (POST with URL in body) 645|Step 1: HTTP Request → fetch page content 646|Step 2: Code node → extract text, chunk into ~500 token segments 647|Step 3: HTTP Request → Ollama API → summarize each chunk 648|Step 4: Code node → combine summaries into digest 649|Step 5: Postgres node → store digest with metadata 650|Step 6: Return digest in webhook response 651| 652| 653|Workflow: Multi-Model Query Router (simplified by LiteLLM — see Phase 7) 654| 655|Trigger: Webhook node (POST with query + complexity hint) 656|Step 1: HTTP Request → LiteLLM (POST http://litellm:4000/v1/chat/completions) 657| - model: complexity parameter ("light", "medium", or "heavy") 658| - LiteLLM handles routing to the correct Ollama instance 659|Step 2: Return response via webhook 660| 661|> With LiteLLM deployed, the Switch node and three separate Ollama endpoints 662|> are replaced by a single HTTP Request node. The routing logic lives in 663|> LiteLLM's config.yaml instead of n8n workflow logic. 664| 665|Workflow: Qdrant Index Updater 666| 667|Trigger: Webhook node (from Gitea push to any watched repo) 668|Step 1: HTTP Request → Gitea API → fetch changed files content 669|Step 2: Code node → chunk text into embedding-sized segments 670|Step 3: HTTP Request → Ollama API (embed endpoint with nomic-embed-text) 671|Step 4: HTTP Request → Qdrant API (PUT /collections/homelab_docs/points) 672|Step 5: Gotify notification: "{n} documents re-indexed" 673| 674| 675|#### Home / Business Workflows 676| 677|Workflow: KitchenOwl Grocery Notification 678| 679|Trigger: Polling (KitchenOwl API) or webhook if supported 680|Step 1: Fetch current shopping list items 681|Step 2: Format as clean text list 682|Step 3: HTTP Request → Gotify → phone push notification 683| 684| 685|Workflow: Donetick Task Reminder 686| 687|Trigger: Cron node (daily at 9:00 AM) 688|Step 1: HTTP Request → Donetick API → fetch tasks due today/overdue 689|Step 2: Format task list 690|Step 3: HTTP Request → Gotify (priority 5) 691| 692| 693|Workflow: Long and Low Crafts Order Pipeline 694| 695|Trigger: Webhook (Etsy webhook or email trigger via IMAP node) 696|Step 1: Parse order details (item, quantity, customer, shipping) 697|Step 2: HTTP Request → Donetick API → create fulfillment task 698|Step 3: HTTP Request → Gotify DM notification 699| - Title: "New L&L Order" 700| - Message: "{item} x{qty} — ship by {date}" 701| - Priority: 7 702| 703| 704|--- 705| 706|## Phase 4 — Paperless-NGX (Document Intelligence) 707| 708|Host: PlausibleDeniability 709|Stack directory: /mnt/docker-ssd/docker/compose/documents/ (new stack) 710|Why fourth: Depends on n8n for automation hooks and Gotify for notifications. 711| 712|### 4.1 — Scaffold Directories 713| 714|bash 715|# Index/data on SSD (search index is write-heavy) 716|sudo mkdir -p /mnt/docker-ssd/docker/appdata/paperless/data 717| 718|# Documents (media) on tank (bulk storage, read-heavy) 719|sudo mkdir -p /mnt/tank/docker/appdata/paperless/media 720| 721|# Consume folder on tank (drop zone for new documents) 722|sudo mkdir -p /mnt/tank/docker/appdata/paperless/consume 723| 724|# Export folder on tank 725|sudo mkdir -p /mnt/tank/docker/appdata/paperless/export 726| 727|# Stack directory 728|sudo mkdir -p /mnt/docker-ssd/docker/compose/documents 729| 730| 731|### 4.2 — Database Init 732| 733|Paperless uses the existing shared-postgres and shared-redis: 734| 735|bash 736|docker exec -i shared-postgres psql -U postgres <<'SQL' 737|CREATE USER paperless WITH PASSWORD 'REPLACE_WITH_GENERATED_PASSWORD'; 738|CREATE DATABASE paperless OWNER paperless; 739|GRANT ALL PRIVILEGES ON DATABASE paperless TO paperless; 740|SQL 741| 742| 743|### 4.3 — docker-compose.yaml 744| 745|Create /mnt/docker-ssd/docker/compose/documents/docker-compose.yaml: 746| 747|yaml 748|name: documents 749| 750|services: 751| paperless: 752| image: ghcr.io/paperless-ngx/paperless-ngx:2.16 753| container_name: paperless 754| restart: unless-stopped 755| ports: 756| - "8000:8000" 757| environment: 758| TZ: ${TZ} 759| # Database 760| PAPERLESS_DBENGINE: postgresql 761| PAPERLESS_DBHOST: shared-postgres 762| PAPERLESS_DBPORT: 5432 763| PAPERLESS_DBNAME: ${PAPERLESS_DB_NAME} 764| PAPERLESS_DBUSER: ${PAPERLESS_DB_USER} 765| PAPERLESS_DBPASS: ${PAPERLESS_DB_PASS} 766| # Redis (using shared-redis) 767| PAPERLESS_REDIS: redis://shared-redis:***@10.5.30.7 1063| 1064|# Stack directory 1065|sudo mkdir -p /opt/monitoring 1066| 1067|# Prometheus data on hdd-2 (has more headroom) 1068|sudo mkdir -p /mnt/hdd-2/prometheus-data 1069|sudo chown 65534:65534 /mnt/hdd-2/prometheus-data # nobody user (Prometheus default) 1070| 1071|# Grafana data on hdd-2 1072|sudo mkdir -p /mnt/hdd-2/grafana-data 1073|sudo chown 472:472 /mnt/hdd-2/grafana-data # grafana user 1074| 1075|# Config directories 1076|sudo mkdir -p /opt/monitoring/provisioning/datasources 1077|sudo mkdir -p /opt/monitoring/provisioning/dashboards 1078| 1079| 1080|### 6.2 — prometheus.yml 1081| 1082|Create /opt/monitoring/prometheus.yml: 1083| 1084|yaml 1085|global: 1086| scrape_interval: 15s 1087| evaluation_interval: 15s 1088| 1089|scrape_configs: 1090| # ------- Local (N.O.M.A.D.) ------- 1091| - job_name: 'nomad-node' 1092| static_configs: 1093| - targets: ['node-exporter:9100'] 1094| labels: 1095| host: 'nomad' 1096| 1097| # ------- PlausibleDeniability ------- 1098| - job_name: 'pd-netdata' 1099| metrics_path: /api/v1/allmetrics 1100| params: 1101| format: [prometheus] 1102| static_configs: 1103| - targets: ['10.5.1.X:19999'] # Replace with PD's IP 1104| labels: 1105| host: 'plausible-deniability' 1106| 1107| # ------- Serenity ------- 1108| - job_name: 'serenity-netdata' 1109| metrics_path: /api/v1/allmetrics 1110| params: 1111| format: [prometheus] 1112| static_configs: 1113| - targets: ['10.5.30.5:19999'] 1114| labels: 1115| host: 'serenity' 1116| 1117| # ------- Gotify ------- 1118| # Gotify exposes /health but no Prometheus endpoint natively. 1119| # Use blackbox exporter or just rely on Uptime Kuma. 1120| 1121| # ------- n8n ------- 1122| # n8n doesn't expose Prometheus metrics by default. 1123| # Monitor via container resource metrics from Netdata. 1124| 1125| # -------- Add more targets as needed -------- 1126| # When node-exporter is installed on PD and Serenity: 1127| # - job_name: 'pd-node' 1128| # static_configs: 1129| # - targets: ['10.5.1.X:9100'] 1130| # labels: 1131| # host: 'plausible-deniability' 1132| # 1133| # - job_name: 'serenity-node' 1134| # static_configs: 1135| # - targets: ['10.5.30.5:9100'] 1136| # labels: 1137| # host: 'serenity' 1138| 1139| 1140|> Netdata as a Prometheus target: Both PD and Serenity already run Netdata. Netdata has a built-in Prometheus exporter at /api/v1/allmetrics?format=prometheus. This gives you CPU, memory, disk, network, and ZFS metrics without installing node-exporter on those hosts. 1141| 1142|### 6.3 — Grafana Provisioning 1143| 1144|Create /opt/monitoring/provisioning/datasources/prometheus.yml: 1145| 1146|yaml 1147|apiVersion: 1 1148| 1149|datasources: 1150| - name: Prometheus 1151| type: prometheus 1152| access: proxy 1153| url: http://prometheus:9090 1154| isDefault: true 1155| editable: true 1156| 1157| 1158|Create /opt/monitoring/provisioning/dashboards/dashboards.yml: 1159| 1160|yaml 1161|apiVersion: 1 1162| 1163|providers: 1164| - name: 'default' 1165| orgId: 1 1166| folder: '' 1167| type: file 1168| disableDeletion: false 1169| editable: true 1170| options: 1171| path: /var/lib/grafana/dashboards 1172| foldersFromFilesStructure: false 1173| 1174| 1175|### 6.4 — docker-compose.yaml 1176| 1177|Create /opt/monitoring/docker-compose.yaml: 1178| 1179|yaml 1180|name: monitoring 1181| 1182|services: 1183| prometheus: 1184| image: prom/prometheus:v3.4.0 1185| container_name: prometheus 1186| restart: unless-stopped 1187| ports: 1188| - "9090:9090" 1189| command: 1190| - '--config.file=/etc/prometheus/prometheus.yml' 1191| - '--storage.tsdb.path=/prometheus' 1192| - '--storage.tsdb.retention.time=90d' 1193| - '--web.enable-lifecycle' 1194| volumes: 1195| - /opt/monitoring/prometheus.yml:/etc/prometheus/prometheus.yml:ro 1196| - /mnt/hdd-2/prometheus-data:/prometheus 1197| networks: 1198| - monitoring 1199| 1200| grafana: 1201| image: grafana/grafana:13.0.1 1202| container_name: grafana 1203| restart: unless-stopped 1204| ports: 1205| - "3000:3000" 1206| environment: 1207| TZ: ${TZ} 1208| GF_SECURITY_ADMIN_USER: ${GF_ADMIN_USER} 1209| GF_SECURITY_ADMIN_PASSWORD: ${GF_ADMIN_PASS} 1210| GF_SERVER_ROOT_URL: https://${GF_HOST}/ 1211| volumes: 1212| - /mnt/hdd-2/grafana-data:/var/lib/grafana 1213| - /opt/monitoring/provisioning:/etc/grafana/provisioning:ro 1214| networks: 1215| - monitoring 1216| 1217| node-exporter: 1218| image: prom/node-exporter:v1.9.0 1219| container_name: node-exporter 1220| restart: unless-stopped 1221| ports: 1222| - "9100:9100" 1223| command: 1224| - '--path.procfs=/host/proc' 1225| - '--path.sysfs=/host/sys' 1226| - '--path.rootfs=/rootfs' 1227| - '--collector.filesystem.mount-points-exclude=^/(sys|proc|dev|host|etc)($$|/)' 1228| volumes: 1229| - /proc:/host/proc:ro 1230| - /sys:/host/sys:ro 1231| - /:/rootfs:ro 1232| networks: 1233| - monitoring 1234| 1235|networks: 1236| monitoring: 1237| driver: bridge 1238| 1239| 1240|> Image notes: 1241|> - prom/prometheus:v3.4.0 — Prometheus v3 is the current major line. Do not use :latest — it still resolves to 2.x due to a known tagging issue. Always use an explicit v3.x.y tag. 1242|> - grafana/grafana:latest — use the moving stable image by default here; only pin Grafana if a breakage or compatibility reason is documented. 1243|> - prom/node-exporter:v1.9.0 — check releases. 1244| 1245|### 6.5 — .env.example 1246| 1247|bash 1248|# /opt/monitoring/.env.example 1249| 1250|TZ=America/New_York 1251| 1252|# Grafana 1253|GF_ADMIN_USER=admin 1254|GF_ADMIN_PASS=CHANGE_ME 1255|GF_HOST=grafana.paccoco.com 1256| 1257| 1258|### 6.6 — Pangolin Configuration 1259| 1260|Grafana runs on N.O.M.A.D., not PD where the main Newt agent lives. You have two options: 1261| 1262|Option A — Route via N.O.M.A.D.'s existing Newt (recommended if already connected) 1263| 1264|N.O.M.A.D. already has a Newt container from the Project N.O.M.A.D. setup. If it's connected to your Pangolin VPS, just add a new resource in the Pangolin dashboard pointing to http://grafana:3000 or http://10.5.30.7:3000. 1265| 1266|Option B — Add a dedicated Newt to the monitoring stack 1267| 1268|If N.O.M.A.D.'s existing Newt is not connected to Pangolin (or is a separate Pangolin instance), add Newt to the monitoring compose: 1269| 1270|yaml 1271| newt: 1272| image: ghcr.io/fosrl/newt:latest 1273| container_name: monitoring-newt 1274| restart: unless-stopped 1275| environment: 1276| PANGOLIN_ENDPOINT: ${PANGOLIN_ENDPOINT} 1277| NEWT_ID: ${NEWT_ID} 1278| NEWT_SECRET: ${NEWT_SECRET} 1279| networks: 1280| - monitoring 1281| 1282| 1283|Then in Pangolin dashboard: 1284| 1285|| Field | Value | 1286||-------|-------| 1287|| Domain | grafana.paccoco.com | 1288|| Scheme | http | 1289|| Host | grafana (Docker DNS via shared network) | 1290|| Port | 3000 | 1291| 1292|Option C — Direct IP routing (simplest, no Newt needed) 1293| 1294|If Pangolin's Newt on PD can reach N.O.M.A.D. by LAN IP (they're on the same subnet): 1295| 1296|| Field | Value | 1297||-------|-------| 1298|| Domain | grafana.paccoco.com | 1299|| Scheme | http | 1300|| Host | 10.5.30.7 | 1301|| Port | 3000 | 1302| 1303|> Decision point: Check if N.O.M.A.D.'s existing Newt is connected to your Pangolin instance before deploying. Run docker ps --filter name=newt on N.O.M.A.D. to verify. If it's running and connected, Option A is zero-effort. If not, Option C is the simplest fallback. 1304| 1305|### 6.7 — Validate & Deploy 1306| 1307|bash 1308|ssh nomad@10.5.30.7 1309| 1310|cd /opt/monitoring 1311|cp .env.example .env 1312|nano .env # Fill in real values 1313| 1314|docker compose --env-file .env config 1315|docker compose --env-file .env up -d 1316| 1317| 1318|### 6.8 — Post-Deploy Verification 1319| 1320|bash 1321|# All three containers running? 1322|docker ps --filter name=prometheus --filter name=grafana --filter name=node-exporter 1323| 1324|# Prometheus scraping targets? 1325|curl -s http://localhost:9090/api/v1/targets | python3 -m json.tool | head -40 1326| 1327|# Grafana UI accessible? 1328|curl -s -o /dev/null -w "%{http_code}" http://localhost:3000/ 1329|# Expected: 200 or 302 1330| 1331|# Node exporter metrics flowing? 1332|curl -s http://localhost:9100/metrics | head -10 1333| 1334| 1335|### 6.9 — Recommended Dashboards 1336| 1337|Import these from Grafana Dashboard Library: 1338| 1339|| Dashboard | ID | Purpose | 1340||-----------|----|---------| 1341|| Node Exporter Full | 1860 | System metrics for N.O.M.A.D. | 1342|| Docker Container Stats | 893 | Container resource usage | 1343|| Netdata via Prometheus | (search) | PD and Serenity system metrics | 1344| 1345|To import: Grafana → Dashboards → New → Import → Enter dashboard ID. 1346| 1347|### 6.10 — Metrics Targets Summary 1348| 1349|| Target | Host | Method | Endpoint | 1350||--------|------|--------|----------| 1351|| N.O.M.A.D. system | localhost | node-exporter | node-exporter:9100 | 1352|| PD system | 10.5.1.X | Netdata Prometheus | 10.5.1.X:19999/api/v1/allmetrics | 1353|| Serenity system | 10.5.30.5 | Netdata Prometheus | 10.5.30.5:19999/api/v1/allmetrics | 1354|| ZFS pools | via Netdata | Netdata exports ZFS metrics | Included in Netdata scrape | 1355|| Container stats | via Netdata | Netdata exports cgroup metrics | Included in Netdata scrape | 1356|| Plex streams | Tautulli | n8n polling → Postgres | Via n8n workflow (Phase 3) | 1357|| qBit stats | qBittorrent API | n8n polling → Postgres | Via n8n workflow (Phase 3) | 1358|| Tailscale latency | Tailscale API | n8n polling → Postgres | Via n8n workflow (Phase 3) | 1359| 1360|### 6.11 — n8n Integration (Grafana → n8n alert webhook) 1361| 1362|In Grafana → Alerting → Contact Points, create a webhook contact point: 1363| 1364|| Field | Value | 1365||-------|-------| 1366|| Name | n8n-alerts | 1367|| Type | Webhook | 1368|| URL | https://n8n.paccoco.com/webhook/grafana-alert | 1369|| HTTP Method | POST | 1370| 1371|Then create alert rules for: 1372|- ZFS pool utilization > 85% 1373|- Container memory > 90% of limit 1374|- Host CPU sustained > 90% for 5 minutes 1375|- Disk I/O latency spikes 1376| 1377|These fire into the "Grafana Alert Remediation" n8n workflow (see Phase 3). 1378| 1379|--- 1380| 1381|## Phase 7 — LiteLLM (AI Gateway) 1382| 1383|Host: PlausibleDeniability 1384|Stack directory: /mnt/docker-ssd/docker/compose/ai/ (same stack as Qdrant) 1385|Why: Replaces manual multi-model routing with a unified OpenAI-compatible API. Every app (OpenWebUI, Continue.dev, n8n, Paperless summarization) points at one endpoint instead of juggling three Ollama IPs. 1386| 1387|### 7.1 — What LiteLLM Does 1388| 1389|LiteLLM sits in front of your three Ollama instances and presents a single OpenAI-compatible API at http://litellm:4000. It handles: 1390|

Model routing — current reliable default is N.O.M.A.D.-local Ollama for background/automation paths, with PD kept available for shared light-tier use and Rocinante treated as optional/manual heavy capacity.
Failover — avoid designs that require Rocinante to be online; degrade to N.O.M.A.D.-local or explicitly operator-invoked PD paths instead of automatic dependence on a personal PC.
Load balancing — distribute requests across instances running the same model when you have multiple dependable backends, but do not assume Rocinante qualifies as one. 1394|- Usage tracking — logs token counts, latency, and costs per model/user via its built-in database 1395| 1396|### 7.2 — Scaffold Directories 1397| 1398|bash 1399|# Config on SSD (SQLite DB for usage tracking) 1400|sudo mkdir -p /mnt/docker-ssd/docker/appdata/litellm 1401| 1402|# LiteLLM config file 1403|sudo mkdir -p /mnt/docker-ssd/docker/compose/ai/litellm 1404| 1405| 1406|### 7.3 — LiteLLM Config 1407| 1408|Create /mnt/docker-ssd/docker/compose/ai/litellm/config.yaml: 1409| 1410|yaml 1411|model_list: 1412| # ---- ROCINANTE (RTX 4090, 24GB) — heavy reasoning ---- 1413| - model_name: "heavy" 1414| litellm_params: 1415| model: "ollama/qwen3:32b" 1416| api_base: "http://10.5.1.ROCINANTE:11434" 1417| timeout: 300 1418| stream_timeout: 300 1419| model_info: 1420| description: "Heavy reasoning, long context, complex code" 1421| 1422| - model_name: "heavy" 1423| litellm_params: 1424| model: "ollama/deepseek-r1:32b" 1425| api_base: "http://10.5.1.ROCINANTE:11434" 1426| timeout: 300 1427| stream_timeout: 300 1428| model_info: 1429| description: "Deep reasoning fallback on ROCINANTE" 1430| 1431| # ---- PlausibleDeniability (RTX 2080 Ti, 11GB) — general ---- 1432| - model_name: "medium" 1433| litellm_params: 1434| model: "ollama/qwen2.5:14b" 1435| api_base: "http://host.docker.internal:11434" 1436| timeout: 120 1437| stream_timeout: 120 1438| model_info: 1439| description: "General homelab assistant, RAG queries" 1440| 1441| # ---- N.O.M.A.D. (GTX 1080, 8GB) — lightweight ---- 1442| - model_name: "light" 1443| litellm_params: 1444| model: "ollama/phi4" 1445| api_base: "http://10.5.30.7:11434" 1446| timeout: 60 1447| stream_timeout: 60 1448| model_info: 1449| description: "Fast local inference, lightweight tasks" 1450| 1451| - model_name: "light" 1452| litellm_params: 1453| model: "ollama/llama3.2:3b" 1454| api_base: "http://10.5.30.7:11434" 1455| timeout: 60 1456| stream_timeout: 60 1457| model_info: 1458| description: "Ultra-light fallback on N.O.M.A.D." 1459| 1460| # ---- Embeddings ---- 1461| - model_name: "embed" 1462| litellm_params: 1463| model: "ollama/nomic-embed-text" 1464| api_base: "http://host.docker.internal:11434" 1465| model_info: 1466| description: "Text embeddings for RAG pipeline" 1467| 1468| # ---- Direct model access (bypass routing) ---- 1469| # These let you request a specific model by its full name 1470| - model_name: "ollama/qwen3:32b" 1471| litellm_params: 1472| model: "ollama/qwen3:32b" 1473| api_base: "http://10.5.1.ROCINANTE:11434" 1474| 1475| - model_name: "ollama/qwen2.5:14b" 1476| litellm_params: 1477| model: "ollama/qwen2.5:14b" 1478| api_base: "http://host.docker.internal:11434" 1479| 1480| - model_name: "ollama/phi4" 1481| litellm_params: 1482| model: "ollama/phi4" 1483| api_base: "http://10.5.30.7:11434" 1484| 1485|litellm_settings: 1486| drop_params: true 1487| set_verbose: false 1488| request_timeout: 300 1489| num_retries: 2 1490| retry_after: 5 1491| allowed_fails: 3 1492| cooldown_time: 60 1493| 1494|general_settings: 1495| master_key: "os.environ/LITELLM_MASTER_KEY" 1496| 1497| 1498|> DEPLOYED NOTE (2026-05-05): Do NOT add database_url to general_settings — newer LiteLLM versions include Prisma ORM that requires PostgreSQL, and adding any database_url causes a crash loop. Without it, LiteLLM runs in config-only mode which is fine for homelab use. The master_key uses os.environ/LITELLM_MASTER_KEY syntax to read from the container's environment variable (set via .env → docker compose). 1499| 1500|> Routing explained: Multiple entries with the same model_name (like "heavy") enable load balancing and failover within that tier. When you request model heavy, LiteLLM picks the healthiest deployment. The direct-access entries (like ollama/qwen3:32b) let you bypass the tier system and target a specific model when needed. 1501| 1502|> Replace IPs: Update 10.5.1.ROCINANTE with ROCINANTE's actual LAN or Tailscale IP. PD uses host.docker.internal since LiteLLM runs on PD alongside Ollama. 1503| 1504|### 7.4 — docker-compose.yaml (updated ai stack) 1505| 1506|Update /mnt/docker-ssd/docker/compose/ai/docker-compose.yaml to add LiteLLM: 1507| 1508|yaml 1509|name: ai 1510| 1511|services: 1512| qdrant: 1513| image: qdrant/qdrant:v1.14.0 1514| container_name: qdrant 1515| restart: unless-stopped 1516| ports: 1517| - "6333:6333" # REST API 1518| - "6334:6334" # gRPC 1519| environment: 1520| TZ: ${TZ} 1521| QDRANT__SERVICE__GRPC_PORT: 6334 1522| QDRANT__STORAGE__STORAGE_PATH: /qdrant/storage 1523| QDRANT__STORAGE__SNAPSHOTS_PATH: /qdrant/snapshots 1524| volumes: 1525| - /mnt/docker-ssd/docker/appdata/qdrant/storage:/qdrant/storage 1526| - /mnt/docker-ssd/docker/appdata/qdrant/snapshots:/qdrant/snapshots 1527| networks: 1528| - ai-services 1529| - default 1530| 1531| litellm: 1532| image: ghcr.io/berriai/litellm:main-latest 1533| container_name: litellm 1534| restart: unless-stopped 1535| ports: 1536| - "4000:4000" 1537| environment: 1538| TZ: ${TZ} 1539| LITELLM_MASTER_KEY: ${LITELLM_MASTER_KEY} 1540| volumes: 1541| - /mnt/docker-ssd/docker/compose/ai/litellm/config.yaml:/app/config.yaml:ro 1542| - /mnt/docker-ssd/docker/appdata/litellm:/app/data 1543| command: ["--config", "/app/config.yaml", "--port", "4000"] 1544| extra_hosts: 1545| - "host.docker.internal:host-gateway" 1546| networks: 1547| - ai-services 1548| - default 1549| 1550| reranker: 1551| image: ghcr.io/huggingface/text-embeddings-inference:cpu-latest 1552| container_name: reranker 1553| restart: unless-stopped 1554| ports: 1555| - "8787:80" 1556| environment: 1557| MODEL_ID: ${RERANKER_MODEL} 1558| volumes: 1559| - /mnt/docker-ssd/docker/appdata/reranker:/data 1560| networks: 1561| - ai-services 1562| - default 1563| 1564| whisper: 1565| image: fedirz/faster-whisper-server:latest-cuda 1566| container_name: whisper 1567| restart: unless-stopped 1568| ports: 1569| - "8786:8000" 1570| environment: 1571| TZ: ${TZ} 1572| WHISPER__MODEL: ${WHISPER_MODEL} 1573| WHISPER__DEVICE: cuda 1574| WHISPER__COMPUTE_TYPE: float16 1575| volumes: 1576| - /mnt/docker-ssd/docker/appdata/whisper:/root/.cache/huggingface 1577| deploy: 1578| resources: 1579| reservations: 1580| devices: 1581| - driver: nvidia 1582| count: 1 1583| capabilities: [gpu] 1584| networks: 1585| - ai-services 1586| - default 1587| 1588| # ------------------------------------------------------- 1589| # Ollama and OpenWebUI go here when deployed. 1590| # They share this stack and the default + ai-services networks. 1591| # OpenWebUI → Qdrant at http://qdrant:6333 1592| # OpenWebUI → LiteLLM at http://litellm:4000 (or direct Ollama) 1593| # ------------------------------------------------------- 1594| 1595| # ollama: 1596| # image: ollama/ollama:latest 1597| # container_name: ollama 1598| # ... 1599| 1600| # openwebui: 1601| # image: ghcr.io/open-webui/open-webui:main 1602| # container_name: openwebui 1603| # ... 1604| 1605|networks: 1606| ai-services: 1607| name: ai-services 1608| 1609| 1610|> GPU sharing note: On PD, the RTX 2080 Ti is shared between Plex (hardware transcoding), Immich ML, and Ollama. faster-whisper will also need the GPU when processing audio. These workloads are bursty (not constant), so time-sharing the GPU is viable — but be aware that a large Whisper transcription job will temporarily impact Ollama inference latency. If this becomes an issue, consider running Whisper on N.O.M.A.D.'s GTX 1080 instead and change whisper's Ollama-style routing accordingly. 1611| 1612|> Reranker on CPU: The reranker uses the cpu-latest image variant intentionally — do NOT pin to cpu-1.5 as it has an hf-hub compatibility bug. Reranking is a lightweight operation (scoring ~50 chunks takes <1s on CPU) and doesn't justify competing for GPU VRAM. The model loads via safetensors fallback since no ONNX files exist for bge-reranker-v2-m3. Max batch size is 4 on CPU. If you find latency is an issue, a GPU variant exists (ghcr.io/huggingface/text-embeddings-inference:turing-latest for your 2080 Ti). 1613| 1614|### 7.5 — .env.example (updated ai stack) 1615| 1616|bash 1617|# /mnt/docker-ssd/docker/compose/ai/.env.example 1618| 1619|TZ=America/New_York 1620| 1621|# LiteLLM 1622|LITELLM_MASTER_KEY=sk-CHANGE_ME 1623| 1624|# Reranker 1625|RERANKER_MODEL=BAAI/bge-reranker-v2-m3 1626| 1627|# Whisper 1628|WHISPER_MODEL=Systran/faster-distil-whisper-large-v3 1629| 1630| 1631|### 7.6 — Validate & Deploy 1632| 1633|bash 1634|cd /mnt/docker-ssd/docker/compose/ai 1635|cp .env.example .env 1636|nano .env # Set LITELLM_MASTER_KEY=$(openssl rand -hex 32) 1637| 1638|docker compose --env-file .env config 1639|docker compose --env-file .env up -d 1640| 1641| 1642|### 7.7 — Post-Deploy Verification 1643| 1644|bash 1645|# All containers running? 1646|docker ps --filter name=qdrant --filter name=litellm --filter name=reranker --filter name=whisper 1647| 1648|# LiteLLM health? 1649|curl -s http://localhost:4000/health 1650| 1651|# LiteLLM can see all models? 1652|curl -s http://localhost:4000/v1/models \ 1653| -H "Authorization: Bearer *** | python3 -m json.tool 1654| 1655|# Test a chat completion through LiteLLM 1656|curl -s http://localhost:4000/v1/chat/completions \ 1657| -H "Authorization: Bearer *** \ 1658| -H "Content-Type: application/json" \ 1659| -d '{ 1660| "model": "medium", 1661| "messages": [{"role": "user", "content": "Hello, which model are you?"}] 1662| }' | python3 -m json.tool 1663| 1664|# Test embeddings through LiteLLM 1665|curl -s http://localhost:4000/v1/embeddings \ 1666| -H "Authorization: Bearer *** \ 1667| -H "Content-Type: application/json" \ 1668| -d '{ 1669| "model": "embed", 1670| "input": "test embedding" 1671| }' | python3 -m json.tool 1672| 1673| 1674|### 7.8 — Integration Updates 1675| 1676|With LiteLLM deployed, all apps should point at http://litellm:4000 (within the ai-services network) or http://PD_IP:4000 (from external hosts) instead of direct Ollama endpoints. 1677| 1678|OpenWebUI: Settings → Connections → add OpenAI-compatible endpoint: 1679|- URL: http://litellm:4000/v1 1680|- API Key: your LITELLM_MASTER_KEY 1681|- This gives OpenWebUI access to all models across all three machines through one connection 1682| 1683|Continue.dev: Update ~/.continue/config.json: 1684|json 1685|{ 1686| "models": [ 1687| { 1688| "title": "Heavy (ROCINANTE)", 1689| "provider": "openai", 1690| "model": "heavy", 1691| "apiBase": "http://PD_TAILSCALE_IP:4000/v1", 1692| "apiKey": "YOUR_MASTER_KEY" 1693| }, 1694| { 1695| "title": "Medium (PD)", 1696| "provider": "openai", 1697| "model": "medium", 1698| "apiBase": "http://PD_TAILSCALE_IP:4000/v1", 1699| "apiKey": "YOUR_MASTER_KEY" 1700| } 1701| ], 1702| "tabAutocompleteModel": { 1703| "title": "Light (N.O.M.A.D.)", 1704| "provider": "openai", 1705| "model": "light", 1706| "apiBase": "http://PD_TAILSCALE_IP:4000/v1", 1707| "apiKey": "YOUR_MASTER_KEY" 1708| } 1709|} 1710| 1711| 1712|> Key change: Continue.dev now uses provider: "openai" instead of provider: "ollama" and points at LiteLLM. One IP to remember, and if you add or move models between machines, you only update config.yaml — not every app. 1713| 1714|n8n workflows: All HTTP Request nodes that call Ollama directly should be updated: 1715|- Old: http://10.5.30.7:11434/api/generate (N.O.M.A.D.) 1716|- New: http://litellm:4000/v1/chat/completions with model: "light" 1717|- n8n is on the ai-services network, so it reaches LiteLLM by Docker DNS 1718| 1719|n8n Multi-Model Query Router workflow: This workflow is now simplified dramatically — instead of a Switch node routing to three different Ollama IPs, it becomes a single HTTP Request node that passes the model tier name (light, medium, heavy) to LiteLLM and lets the gateway handle routing. The Switch node logic can be removed entirely. 1720| 1721|--- 1722| 1723|## Phase 8 — Reranker (RAG Quality) 1724| 1725|Host: PlausibleDeniability (deployed as part of the ai stack in Phase 7) 1726|Service: reranker container (already in the compose above) 1727|Why: Dramatically improves RAG answer quality by filtering out noisy retrieval results before they reach the LLM. 1728| 1729|### 8.1 — How Reranking Works 1730| 1731|Without a reranker, your RAG pipeline does: 1732| 1733|Query → embed → Qdrant top-10 by vector similarity → all 10 chunks go to LLM 1734| 1735| 1736|The problem: vector similarity often returns "close but irrelevant" chunks. The LLM gets noisy context and hallucinates. 1737| 1738|With a reranker: 1739| 1740|Query → embed → Qdrant top-50 by vector similarity → reranker scores each (query, chunk) pair → top-5 by relevance go to LLM 1741| 1742| 1743|The reranker is a cross-encoder that reads the full query AND each chunk together, producing a much more accurate relevance score than vector distance alone. It over-retrieves cheaply from Qdrant, then precisely filters. 1744| 1745|### 8.2 — Scaffold Directories 1746| 1747|bash 1748|# Model cache on SSD (reranker model is ~1.1GB, downloaded on first start) 1749|sudo mkdir -p /mnt/docker-ssd/docker/appdata/reranker 1750| 1751| 1752|### 8.3 — Post-Deploy Verification 1753| 1754|bash 1755|# Container running? 1756|docker ps --filter name=reranker 1757| 1758|# Health check? 1759|curl -s http://localhost:8787/health 1760| 1761|# Test reranking 1762|curl -s http://localhost:8787/rerank \ 1763| -H "Content-Type: application/json" \ 1764| -d '{ 1765| "query": "How do I restart a Docker container?", 1766| "texts": [ 1767| "Use docker restart <container_name> to restart a running container.", 1768| "Docker was founded in 2013 by Solomon Hykes.", 1769| "The docker compose down command stops and removes containers.", 1770| "Kubernetes pods can be restarted by deleting them." 1771| ] 1772| }' | python3 -m json.tool 1773|# Expected: the first text scores highest, second scores lowest 1774| 1775| 1776|### 8.4 — n8n RAG Pipeline Integration 1777| 1778|Update the "Qdrant Index Updater" and any RAG query workflows to include a reranking step. 1779| 1780|Updated RAG Query Workflow (for OpenWebUI or any n8n-based query): 1781| 1782|Trigger: Webhook node (POST with query) 1783|Step 1: HTTP Request → LiteLLM /v1/embeddings 1784| - model: "embed" 1785| - input: query text 1786|Step 2: HTTP Request → Qdrant API (POST /collections/{name}/points/search) 1787| - vector: embedding from step 1 1788| - limit: 50 (over-retrieve) 1789|Step 3: HTTP Request → Reranker (POST http://reranker:80/rerank) 1790| - query: original query text 1791| - texts: array of 50 chunk texts from Qdrant results 1792|Step 4: Code node → take top 5 by reranker score, format as context 1793|Step 5: HTTP Request → LiteLLM /v1/chat/completions 1794| - model: "medium" (or "heavy" for complex queries) 1795| - messages: system prompt with top-5 context + user query 1796|Step 6: Return response via webhook 1797| 1798| 1799|> Key difference from the original plan: Step 2 now retrieves 50 results instead of 10, and Step 3 (reranking) filters down to the best 5. This "over-retrieve then rerank" pattern is the standard approach for production RAG systems. 1800| 1801|--- 1802| 1803|## Phase 9 — faster-whisper (Speech-to-Text) 1804| 1805|Host: PlausibleDeniability (deployed as part of the ai stack in Phase 7) 1806|Service: whisper container (already in the compose above) 1807|Why: Replaces shelved Scriberr with an OpenAI-compatible STT API. No SQLite dependency, no ZFS/ACL issues. 1808| 1809|### 9.1 — Scaffold Directories 1810| 1811|bash 1812|# Model cache on SSD (Whisper models are 1-3GB) 1813|sudo mkdir -p /mnt/docker-ssd/docker/appdata/whisper 1814| 1815| 1816|### 9.2 — Post-Deploy Verification 1817| 1818|bash 1819|# Container running? 1820|docker ps --filter name=whisper 1821| 1822|# Health check? 1823|curl -s http://localhost:8786/health 1824| 1825|# Test transcription with a sample audio file 1826|curl -s http://localhost:8786/v1/audio/transcriptions \ 1827| -F "file=@/path/to/test-audio.wav" \ 1828| -F "model=Systran/faster-distil-whisper-large-v3" \ 1829| | python3 -m json.tool 1830| 1831| 1832|### 9.3 — n8n Integration Workflows 1833| 1834|Workflow: Voice Note → Text → Summary 1835| 1836|Trigger: Webhook node (POST with audio file in body) 1837|Step 1: HTTP Request → Whisper (POST http://whisper:8000/v1/audio/transcriptions) 1838| - Multipart form with audio file 1839| - response_format: "json" 1840|Step 2: HTTP Request → LiteLLM /v1/chat/completions 1841| - model: "medium" 1842| - Prompt: "Summarize this voice note concisely: {transcript}" 1843|Step 3: HTTP Request → Gotify → push summary to phone 1844|Step 4: (Optional) Postgres node → log transcript and summary 1845| 1846| 1847|Workflow: Audio File → Paperless Document 1848| 1849|Trigger: Webhook or filesystem watcher 1850|Step 1: HTTP Request → Whisper → get transcript 1851|Step 2: Code node → format transcript as text document 1852|Step 3: HTTP Request → Paperless API (POST /api/documents/post_document/) 1853| - Upload transcript as .txt 1854| - Tag: "transcription" 1855|Step 4: Gotify notification: "Audio transcribed and filed: {title}" 1856| 1857| 1858|Home Assistant Voice Integration: 1859|If you want HA voice commands, Whisper can serve as the STT backend: 1860|1. In Home Assistant → Settings → Voice Assistants 1861|2. Add speech-to-text provider: "Whisper" at http://PD_IP:8786 1862|3. Pair with Piper TTS (future addition) for full voice assistant loop 1863| 1864|### 9.4 — GPU vs CPU Considerations 1865| 1866|The compose file above uses the GPU variant with CUDA. If GPU contention with Ollama becomes an issue: 1867| 1868|Option A — CPU fallback on PD: 1869|Change the image from latest-cuda to fedirz/faster-whisper-server:latest-cpu and remove the deploy.resources block and the CUDA environment variables (WHISPER__DEVICE, WHISPER__COMPUTE_TYPE). Transcription will be slower (~4x real-time instead of ~20x) but won't compete for VRAM. 1870| 1871|Option B — Move to N.O.M.A.D.: 1872|Run Whisper on N.O.M.A.D.'s GTX 1080 which is less contested. Add it to N.O.M.A.D.'s compose or run standalone. N.O.M.A.D.'s 8GB VRAM handles Whisper models comfortably since phi4 doesn't fill it. 1873| 1874|--- 1875| 1876|## Additional Tools Setup 1877| 1878|### Continue.dev (Local AI Code Completion) 1879| 1880|1. Install the Continue extension in VS Code 1881|2. Create/edit ~/.continue/config.json: 1882| 1883|With LiteLLM (recommended — see Phase 7): 1884|json 1885|{ 1886| "models": [ 1887| { 1888| "title": "Heavy (ROCINANTE)", 1889| "provider": "openai", 1890| "model": "heavy", 1891| "apiBase": "http://PD_TAILSCALE_IP:4000/v1", 1892| "apiKey": "YOUR_LITELLM_MASTER_KEY" 1893| }, 1894| { 1895| "title": "Medium (PD)", 1896| "provider": "openai", 1897| "model": "medium", 1898| "apiBase": "http://PD_TAILSCALE_IP:4000/v1", 1899| "apiKey": "YOUR_LITELLM_MASTER_KEY" 1900| } 1901| ], 1902| "tabAutocompleteModel": { 1903| "title": "Light (N.O.M.A.D.)", 1904| "provider": "openai", 1905| "model": "light", 1906| "apiBase": "http://PD_TAILSCALE_IP:4000/v1", 1907| "apiKey": "YOUR_LITELLM_MASTER_KEY" 1908| } 1909|} 1910| 1911| 1912|> One IP, one API key, all three machines. If you add or move models, update LiteLLM's config.yaml — not every app. 1913| 1914|Without LiteLLM (direct Ollama, if Phase 7 is not yet deployed): 1915|json 1916|{ 1917| "models": [ 1918| { 1919| "title": "N.O.M.A.D. - qwen2.5:3b", 1920| "provider": "ollama", 1921| "model": "qwen2.5:3b", 1922| "apiBase": "http://NOMAD_LOCAL_OR_TAILSCALE_IP:11434" 1923| }, 1924| { 1925| "title": "PD - shared light tier (manual / fallback)", 1926| "provider": "ollama", 1927| "model": "qwen2.5:14b", 1928| "apiBase": "http://PD_TAILSCALE_IP:11434" 1929| }, 1930| { 1931| "title": "ROCINANTE - opportunistic heavy", 1932| "provider": "ollama", 1933| "model": "qwen3:32b", 1934| "apiBase": "http://ROCINANTE_TAILSCALE_IP:11434" 1935| } 1930| ], 1931| "tabAutocompleteModel": { 1932| "title": "N.O.M.A.D. - phi4", 1933| "provider": "ollama", 1934| "model": "phi4", 1935| "apiBase": "http://NOMAD_TAILSCALE_IP:11434" 1936| } 1937|} 1938| 1939| 1940|### Obsidian + Gitea Sync 1941| 1942|1. In Obsidian, install the "Obsidian Git" community plugin 1943|2. Initialize your vault as a git repo: 1944| bash 1945| cd /path/to/obsidian/vault 1946| git init 1947| git remote add origin http://PD_IP:3000/fizzlepoof/obsidian-vault.git 1948| 1949|3. In Obsidian Git settings: 1950| - Auto backup interval: 10 minutes 1951| - Pull on startup: enabled 1952|4. Create the repo in Gitea first: http://PD_IP:3000 → New Repository → obsidian-vault 1953|5. Add a Gitea webhook to trigger the n8n "Qdrant Index Updater" workflow (see Phase 3) 1954| 1955|### Homepage Ollama Widget 1956| 1957|Add to your Homepage configuration (/mnt/tank/docker/appdata/homepage/services.yaml): 1958| 1959|yaml 1960|- AI: 1961| - Ollama (PD): 1962| icon: ollama.svg 1963| href: http://PD_IP:11434 1964| widget: 1965| type: ollama 1966| url: http://PD_IP:11434 1967| - Ollama (ROCINANTE): 1968| icon: ollama.svg 1969| href: http://ROCINANTE_IP:11434 1970| widget: 1971| type: ollama 1972| url: http://ROCINANTE_IP:11434 1973| 1974| 1975|--- 1976| 1977|## Deployment Order Summary 1978| 1979|``` 1980|Week 1: Phase 1 — Gotify 1981| └─ Deploy, create app tokens, install phone app 1982| └─ Test: send manual notification via API 1983| 1984|Week 1: Phase 2 — Qdrant 1985| └─ Deploy, verify REST API 1986| └─ Create initial collections (empty, ready for n8n) 1987| 1988|Week 2: Phase 3 — n8n 1989| └─ Deploy, create admin account 1990| └─ Build workflows incrementally: 1991| Day 1: Gitea commit → Gotify (simplest, proves the pipeline) 1992| Day 2: Sonarr/Radarr → TMDB → Gotify + Discord 1993| Day 3: Tautulli play logging + weekly digest 1994| Day 4: Uptime Kuma enhanced alerts 1995| Day 5: ZFS pool monitoring 1996| Day 6: Multi-model query router 1997| Day 7: Qdrant index updater 1998| 1999|Week 3: Phase 4 — Paperless-NGX 2000| └─ Deploy, ingest test documents 2001|

66 KiB Raw Blame History

66 KiB

Raw Blame History