truenas-stacks/n8n-workflows/README.md

# n8n AI Workflows for Homelab

## Workflows Included

| # | Workflow | Trigger | What It Does |
|---|---------|---------|--------------|
| 01 | Grafana Alert → AI → Gotify | `POST /webhook/grafana-alert` | Receives Grafana alerts, AI-summarizes, deduplicates within 30 min, suppresses low-priority alerts overnight — pushes to Gotify |
| 02 | RAG Pipeline | `POST /webhook/rag/ingest` + `POST /webhook/rag/query` | Ingest docs into Qdrant, query them with AI-powered answers |
| 03 | Paperless AI Processing | `POST /webhook/paperless/new-document` | Auto-classifies & summarizes new Paperless docs, applies AI-suggested tags back to Paperless, notifies via Gotify |
| 04 | Whisper Transcription | `POST /webhook/transcribe` + `POST /webhook/transcribe/summarize` | Transcribe audio → text, optionally with AI summary; notifies via Gotify when done and saves transcript to Paperless |
| 05 | Paperless → RAG | `POST /webhook/paperless/rag-ingest` | Auto-ingests Paperless documents into Qdrant for RAG search; notifies via Gotify |
| 06 | RSS Digest | Scheduled (every 6h) + `POST /webhook/scrape/summarize` | Fetches RSS feeds, deduplicates via Postgres, AI-summarizes per-feed, pushes to Gotify |
| 07 | Git Commit Summarizer | `POST /webhook/git/push` | Receives git push webhooks, AI-summarizes changes, notifies via Gotify |
| 08 | Class Recording → RAG | `POST /webhook/class/upload` | Transcribes uploaded .wav, extracts key notes, ingests into RAG by class, saves notes to Paperless, notifies via Gotify |
| 14 | Paperless Inbox Reminder | Scheduled (Monday 9 AM) | Queries Paperless for untagged documents; sends Gotify reminder with oldest items listed, or a ✅ clear if inbox is empty |
| 15 | n8n Self-Health Monitor | Scheduled (every 15 min) | Checks n8n execution history for failures in the past hour; deduplicates via Postgres; pushes Gotify alert with workflow name and error summary |
| 16 | NFS Mount Watchdog | Scheduled (every 5 min) | Probes NFS mount points on PD; if missing: runs mount script, re-probes, restarts affected containers (Plex, Audiobookshelf, Immich); notifies via Gotify on heal or escalation |

## How to Import

1. Open n8n at https://n8n.paccoco.com
2. Go to **Workflows** → click the **⋮** menu → **Import from File**
3. Select each `.json` file from this folder
4. Activate the workflow (toggle in top-right)

## Gotify Channel Map

Each workflow uses a dedicated Gotify app credential. Create these apps in Gotify and add matching credentials in n8n → Credentials:

| Gotify App | Used by |
|------------|---------|
| Gotify account | 01 (Grafana Alerts) |
| Paperless | 03 |
| Whisper | 04 |
| Paperless RAG | 05 |
| RSS Feed | 06 |
| Git Commits | 07 |
| Class Recordings | 08 |
| Reminders | 14 |
| n8n Health | 15 |
| Uptime / Infra | 16 |

## One-Time Setup

### Postgres Dedup Tables (required for workflows 01, 15)

```sql
-- Run via: sudo docker exec -it shared-postgres psql -U postgres -d n8n
CREATE TABLE IF NOT EXISTS grafana_alert_dedup (
  fingerprint TEXT PRIMARY KEY,
  last_seen   TIMESTAMPTZ DEFAULT NOW(),
  alert_name  TEXT,
  host        TEXT
);
CREATE TABLE IF NOT EXISTS n8n_health_dedup (
  fingerprint    TEXT PRIMARY KEY,
  last_seen      TIMESTAMPTZ DEFAULT NOW(),
  workflow_names TEXT
);
```

### n8n Environment Variables

Two steps required — both the `.env` file AND the `docker-compose.yaml` `environment:` section must be updated, then n8n restarted with `sudo docker compose --env-file .env down n8n && sudo docker compose --env-file .env up -d n8n`.

**automation/.env** — add these:

| Variable | Used by | How to get |
|----------|---------|------------|
| `PAPERLESS_API_TOKEN` | 04, 08 | Paperless → Settings → API token |
| `PAPERLESS_TAG_TRANSCRIPTION` | 04 | Tag ID from Paperless API |
| `PAPERLESS_TAG_LECTURE_NOTES` | 08 | Tag ID from Paperless API |
| `N8N_API_KEY` | 15 | n8n → Settings → API → Create API Key |
| `LITELLM_API_KEY` | 01, 04, 08 | LiteLLM virtual key (already in .env) |

**automation/docker-compose.yaml** — add to n8n `environment:` section:

```yaml
      N8N_BLOCK_ENV_ACCESS_IN_NODE: "false"
      LITELLM_API_KEY: ${LITELLM_API_KEY}
      PAPERLESS_API_TOKEN: ${PAPERLESS_API_TOKEN}
      PAPERLESS_TAG_TRANSCRIPTION: ${PAPERLESS_TAG_TRANSCRIPTION}
      PAPERLESS_TAG_LECTURE_NOTES: ${PAPERLESS_TAG_LECTURE_NOTES}
      N8N_API_KEY: ${N8N_API_KEY}
```

> **Note:** `N8N_BLOCK_ENV_ACCESS_IN_NODE: "false"` is required for workflows to read `$env.*` variables. Without it, env var access is silently denied even if the variable exists in the container.

### SSH Credential (required for workflow 16)

Workflow 16 SSHes into PD to probe mounts and restart containers. Set up key-based auth:

```bash
# Run on PD — generate a dedicated key for n8n
ssh-keygen -t ed25519 -f ~/.ssh/n8n_watchdog -N ""
cat ~/.ssh/n8n_watchdog.pub >> ~/.ssh/authorized_keys
cat ~/.ssh/n8n_watchdog   # paste this into n8n → Credentials → SSH (Private Key)
```

In n8n: Credentials → New → SSH (Private Key) → host `10.5.1.6`, user `truenas_admin`, paste private key. Name it `PD SSH`.

Workflow 16 also runs `sudo docker restart` and the mount script over SSH. Add a NOPASSWD sudoers rule so it doesn't hang:

```bash
sudo visudo -f /etc/sudoers.d/n8n-watchdog
# Add:
# truenas_admin ALL=(ALL) NOPASSWD: /usr/bin/docker, /bin/bash /mnt/tank/docker/scripts/mount-unraid-nfs.sh
```

### Qdrant Collection (required for workflows 02, 05, 08)

Create the `knowledge_base` collection before using the RAG workflows:

```bash
curl -X PUT http://10.5.1.6:6333/collections/knowledge_base \
  -H 'Content-Type: application/json' \
  -d '{
    "vectors": {
      "size": 768,
      "distance": "Cosine"
    }
  }'
```

(768 dimensions matches `nomic-embed-text`. If you use a different embedding model, adjust the size.)

### Pull nomic-embed-text into Ollama (required for workflows 02, 05, 08)

```bash
curl http://10.5.1.6:11434/api/pull -d '{"name": "nomic-embed-text"}'
```

### Paperless Webhook (required for workflows 03 and 05)

In Paperless-NGX, set up post-consumption webhooks to POST to:
```
https://n8n.paccoco.com/webhook/paperless/new-document
https://n8n.paccoco.com/webhook/paperless/rag-ingest
```
with body: `{"document_id": <id>}`

You can trigger both from the same Paperless event — workflow 03 classifies/tags the document, workflow 05 ingests it into RAG.

### Grafana Webhook (already configured)

Your Grafana contact point is already set to `https://n8n.paccoco.com/webhook/grafana-alert` — workflow 01 will work immediately once activated.

### Git Webhook (required for workflow 07)

In your Gitea/Forgejo/GitHub repo settings, add a webhook:
- URL: `https://n8n.paccoco.com/webhook/git/push`
- Content type: `application/json`
- Events: Push only

Works with Gitea, Forgejo, GitHub, and GitLab webhook payloads.

## Usage Examples

### Ingest a document into RAG:
```bash
curl -X POST https://n8n.paccoco.com/webhook/rag/ingest \
  -H 'Content-Type: application/json' \
  -d '{
    "text": "Your document content here...",
    "source": "my-notes",
    "metadata": {"topic": "homelab"}
  }'
```

### Query your knowledge base:
```bash
curl -X POST https://n8n.paccoco.com/webhook/rag/query \
  -H 'Content-Type: application/json' \
  -d '{"query": "How do I configure Qdrant?"}'
```

### Transcribe audio:
```bash
curl -X POST https://n8n.paccoco.com/webhook/transcribe \
  -F "data=@recording.mp3"
```

### Transcribe + AI summarize:
```bash
curl -X POST https://n8n.paccoco.com/webhook/transcribe/summarize \
  -F "data=@meeting.mp3"
```

### Add a class recording (just drop a file):
```
/mnt/tank/class-recordings/
├── CS101-Java/
│   ├── lecture-01-intro.wav
│   ├── lecture-02-variables.wav
│   └── lecture-03-loops.wav
├── MATH201/
│   └── lecture-01-derivatives.wav
└── ENG102/
    └── essay-workshop.wav
```

Workflow 08 uses a webhook trigger (`POST /webhook/class/upload`), not a folder watcher. Upload the file via curl and include the class name and lecture title as form fields (see curl example below).

> **Note:** The folder structure above shows suggested organization on disk. The workflow does not watch the folder — you must POST to the webhook.

### Query RAG for class-specific content:
```bash
curl -X POST https://n8n.paccoco.com/webhook/rag/query \
  -H 'Content-Type: application/json' \
  -d '{"query": "What did my CS101 lecture say about inheritance in Java?"}'
```

The RAG system stores class name metadata with each chunk, so when you mention a class name in your query it will prioritize results from that class.

### Class Recordings Setup (Workflow 08)

Create class subfolders on PD (already done for CS101-Java):
```bash
sudo mkdir -p /mnt/tank/class-recordings/CS101-Java
sudo mkdir -p /mnt/tank/class-recordings/MATH201
```

Add to n8n's **automation/.env**:
```
PAPERLESS_TAG_LECTURE_NOTES=11
```

Add to n8n's **automation/docker-compose.yaml** under the n8n `environment:` section:
```yaml
      PAPERLESS_TAG_LECTURE_NOTES: ${PAPERLESS_TAG_LECTURE_NOTES}
```

Then restart n8n and import/activate `08-class-recording-rag-v1.1.json`.

To upload a class recording:
```bash
curl -X POST https://n8n.paccoco.com/webhook/class/upload \
  -F "data=@lecture-01-intro.wav" \
  -F "class_name=CS101-Java" \
  -F "lecture_title=lecture-01-intro"
```

### Trigger RSS digest manually:
```bash
curl -X POST https://n8n.paccoco.com/webhook/scrape/summarize
```

### Test git webhook:
```bash
curl -X POST https://n8n.paccoco.com/webhook/git/push \
  -H 'Content-Type: application/json' \
  -d '{
    "ref": "refs/heads/main",
    "pusher": {"name": "john"},
    "repository": {"full_name": "john/my-project"},
    "commits": [
      {"id": "abc1234", "message": "Fix null pointer in UserService", "author": {"name": "john"}, "added": [], "modified": ["src/UserService.java"], "removed": []}
    ]
  }'
```

## Customizing RSS Feeds (Workflow 06)

Edit the "Define RSS Feeds" Code node to add/remove feeds. Default feeds:
- Self-Hosted Show
- r/selfhosted
- r/homelab  
- Hacker News

Add any RSS feed URL you want monitored.