Add n8n workflows 01-08 (with upgrades), 14-16; planning docs; .gitignore
This commit is contained in:
Paccoco
242
n8n-workflows/README.md
Normal file
242
n8n-workflows/README.md
Normal file
@@ -0,0 +1,242 @@
|
||||
# n8n AI Workflows for Homelab
|
||||
|
||||
## Workflows Included
|
||||
|
||||
| # | Workflow | Trigger | What It Does |
|
||||
|---|---------|---------|--------------|
|
||||
| 01 | Grafana Alert → AI → Gotify | `POST /webhook/grafana-alert` | Receives Grafana alerts, AI-summarizes, deduplicates within 30 min, suppresses low-priority alerts overnight — pushes to Gotify |
|
||||
| 02 | RAG Pipeline | `POST /webhook/rag/ingest` + `POST /webhook/rag/query` | Ingest docs into Qdrant, query them with AI-powered answers |
|
||||
| 03 | Paperless AI Processing | `POST /webhook/paperless/new-document` | Auto-classifies & summarizes new Paperless docs, applies AI-suggested tags back to Paperless, notifies via Gotify |
|
||||
| 04 | Whisper Transcription | `POST /webhook/transcribe` + `POST /webhook/transcribe/summarize` | Transcribe audio → text, optionally with AI summary; notifies via Gotify when done and saves transcript to Paperless |
|
||||
| 05 | Paperless → RAG | `POST /webhook/paperless/rag-ingest` | Auto-ingests Paperless documents into Qdrant for RAG search; notifies via Gotify |
|
||||
| 06 | RSS Digest | Scheduled (every 6h) + `POST /webhook/scrape/summarize` | Fetches RSS feeds, deduplicates via Postgres, AI-summarizes per-feed, pushes to Gotify |
|
||||
| 07 | Git Commit Summarizer | `POST /webhook/git/push` | Receives git push webhooks, AI-summarizes changes, notifies via Gotify |
|
||||
| 08 | Class Recording → RAG | Watches `/mnt/data/class-recordings/` | Auto-transcribes new .wav files, extracts key notes, ingests into RAG by class, saves notes to Paperless, notifies via Gotify |
|
||||
| 14 | Paperless Inbox Reminder | Scheduled (Monday 9 AM) | Queries Paperless for untagged documents; sends Gotify reminder with oldest items listed, or a ✅ clear if inbox is empty |
|
||||
| 15 | n8n Self-Health Monitor | Scheduled (every 15 min) | Checks n8n execution history for failures in the past hour; deduplicates via Postgres; pushes Gotify alert with workflow name and error summary |
|
||||
| 16 | NFS Mount Watchdog | Scheduled (every 5 min) | Probes NFS mount points on PD; if missing: runs mount script, re-probes, restarts affected containers (Plex, Audiobookshelf, Immich); notifies via Gotify on heal or escalation |
|
||||
|
||||
## How to Import
|
||||
|
||||
1. Open n8n at https://n8n.paccoco.com
|
||||
2. Go to **Workflows** → click the **⋮** menu → **Import from File**
|
||||
3. Select each `.json` file from this folder
|
||||
4. Activate the workflow (toggle in top-right)
|
||||
|
||||
## Gotify Channel Map
|
||||
|
||||
Each workflow uses a dedicated Gotify app credential. Create these apps in Gotify and add matching credentials in n8n → Credentials:
|
||||
|
||||
| Gotify App | Used by |
|
||||
|------------|---------|
|
||||
| Gotify account | 01 (Grafana Alerts) |
|
||||
| Paperless | 03 |
|
||||
| Whisper | 04 |
|
||||
| Paperless RAG | 05 |
|
||||
| RSS Feed | 06 |
|
||||
| Git Commits | 07 |
|
||||
| Class Recordings | 08 |
|
||||
| Reminders | 14 |
|
||||
| n8n Health | 15 |
|
||||
| Uptime / Infra | 16 |
|
||||
|
||||
## One-Time Setup
|
||||
|
||||
### Postgres Dedup Tables (required for workflows 01, 15)
|
||||
|
||||
```sql
|
||||
-- Run via: sudo docker exec -it shared-postgres psql -U postgres -d n8n
|
||||
CREATE TABLE IF NOT EXISTS grafana_alert_dedup (
|
||||
fingerprint TEXT PRIMARY KEY,
|
||||
last_seen TIMESTAMPTZ DEFAULT NOW(),
|
||||
alert_name TEXT,
|
||||
host TEXT
|
||||
);
|
||||
CREATE TABLE IF NOT EXISTS n8n_health_dedup (
|
||||
fingerprint TEXT PRIMARY KEY,
|
||||
last_seen TIMESTAMPTZ DEFAULT NOW(),
|
||||
workflow_names TEXT
|
||||
);
|
||||
```
|
||||
|
||||
### n8n Environment Variables
|
||||
|
||||
Add these to n8n's `.env` and run `sudo docker compose up -d` to apply:
|
||||
|
||||
| Variable | Used by | How to get |
|
||||
|----------|---------|------------|
|
||||
| `PAPERLESS_API_TOKEN` | 04, 08 | Paperless → Settings → API token |
|
||||
| `PAPERLESS_TAG_TRANSCRIPTION` | 04 | Tag ID from Paperless API |
|
||||
| `PAPERLESS_TAG_LECTURE_NOTES` | 08 | Tag ID from Paperless API |
|
||||
| `N8N_API_KEY` | 15 | n8n → Settings → API → Create API Key |
|
||||
|
||||
### SSH Credential (required for workflow 16)
|
||||
|
||||
Workflow 16 SSHes into PD to probe mounts and restart containers. Set up key-based auth:
|
||||
|
||||
```bash
|
||||
# Run on PD — generate a dedicated key for n8n
|
||||
ssh-keygen -t ed25519 -f ~/.ssh/n8n_watchdog -N ""
|
||||
cat ~/.ssh/n8n_watchdog.pub >> ~/.ssh/authorized_keys
|
||||
cat ~/.ssh/n8n_watchdog # paste this into n8n → Credentials → SSH (Private Key)
|
||||
```
|
||||
|
||||
In n8n: Credentials → New → SSH (Private Key) → host `10.5.1.6`, user `truenas_admin`, paste private key. Name it `PD SSH`.
|
||||
|
||||
Workflow 16 also runs `sudo docker restart` and the mount script over SSH. Add a NOPASSWD sudoers rule so it doesn't hang:
|
||||
|
||||
```bash
|
||||
sudo visudo -f /etc/sudoers.d/n8n-watchdog
|
||||
# Add:
|
||||
# truenas_admin ALL=(ALL) NOPASSWD: /usr/bin/docker, /bin/bash /mnt/tank/docker/scripts/mount-unraid-nfs.sh
|
||||
```
|
||||
|
||||
### Qdrant Collection (required for workflows 02, 05, 08)
|
||||
|
||||
Create the `knowledge_base` collection before using the RAG workflows:
|
||||
|
||||
```bash
|
||||
curl -X PUT http://10.5.1.6:6333/collections/knowledge_base \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d '{
|
||||
"vectors": {
|
||||
"size": 768,
|
||||
"distance": "Cosine"
|
||||
}
|
||||
}'
|
||||
```
|
||||
|
||||
(768 dimensions matches `nomic-embed-text`. If you use a different embedding model, adjust the size.)
|
||||
|
||||
### Pull nomic-embed-text into Ollama (required for workflows 02, 05, 08)
|
||||
|
||||
```bash
|
||||
curl http://10.5.1.6:11434/api/pull -d '{"name": "nomic-embed-text"}'
|
||||
```
|
||||
|
||||
### Paperless Webhook (required for workflows 03 and 05)
|
||||
|
||||
In Paperless-NGX, set up post-consumption webhooks to POST to:
|
||||
```
|
||||
https://n8n.paccoco.com/webhook/paperless/new-document
|
||||
https://n8n.paccoco.com/webhook/paperless/rag-ingest
|
||||
```
|
||||
with body: `{"document_id": <id>}`
|
||||
|
||||
You can trigger both from the same Paperless event — workflow 03 classifies/tags the document, workflow 05 ingests it into RAG.
|
||||
|
||||
### Grafana Webhook (already configured)
|
||||
|
||||
Your Grafana contact point is already set to `https://n8n.paccoco.com/webhook/grafana-alert` — workflow 01 will work immediately once activated.
|
||||
|
||||
### Git Webhook (required for workflow 07)
|
||||
|
||||
In your Gitea/Forgejo/GitHub repo settings, add a webhook:
|
||||
- URL: `https://n8n.paccoco.com/webhook/git/push`
|
||||
- Content type: `application/json`
|
||||
- Events: Push only
|
||||
|
||||
Works with Gitea, Forgejo, GitHub, and GitLab webhook payloads.
|
||||
|
||||
## Usage Examples
|
||||
|
||||
### Ingest a document into RAG:
|
||||
```bash
|
||||
curl -X POST https://n8n.paccoco.com/webhook/rag/ingest \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d '{
|
||||
"text": "Your document content here...",
|
||||
"source": "my-notes",
|
||||
"metadata": {"topic": "homelab"}
|
||||
}'
|
||||
```
|
||||
|
||||
### Query your knowledge base:
|
||||
```bash
|
||||
curl -X POST https://n8n.paccoco.com/webhook/rag/query \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d '{"query": "How do I configure Qdrant?"}'
|
||||
```
|
||||
|
||||
### Transcribe audio:
|
||||
```bash
|
||||
curl -X POST https://n8n.paccoco.com/webhook/transcribe \
|
||||
-F "data=@recording.mp3"
|
||||
```
|
||||
|
||||
### Transcribe + AI summarize:
|
||||
```bash
|
||||
curl -X POST https://n8n.paccoco.com/webhook/transcribe/summarize \
|
||||
-F "data=@meeting.mp3"
|
||||
```
|
||||
|
||||
### Add a class recording (just drop a file):
|
||||
```
|
||||
/mnt/data/class-recordings/
|
||||
├── CS101-Java/
|
||||
│ ├── lecture-01-intro.wav
|
||||
│ ├── lecture-02-variables.wav
|
||||
│ └── lecture-03-loops.wav
|
||||
├── MATH201/
|
||||
│ └── lecture-01-derivatives.wav
|
||||
└── ENG102/
|
||||
└── essay-workshop.wav
|
||||
```
|
||||
|
||||
The workflow detects the class from the subfolder name and the lecture title from the filename.
|
||||
Just drop a `.wav` file into the right class folder and it auto-processes.
|
||||
|
||||
### Query RAG for class-specific content:
|
||||
```bash
|
||||
curl -X POST https://n8n.paccoco.com/webhook/rag/query \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d '{"query": "What did my CS101 lecture say about inheritance in Java?"}'
|
||||
```
|
||||
|
||||
The RAG system stores class name metadata with each chunk, so when you mention a class name in your query it will prioritize results from that class.
|
||||
|
||||
### Class Recordings Setup (Workflow 08)
|
||||
|
||||
Requires n8n volume mount to the NFS share. Add to n8n's docker-compose:
|
||||
```yaml
|
||||
volumes:
|
||||
- /mnt/tank/docker/appdata/n8n:/home/node/.n8n
|
||||
- /mnt/data/class-recordings:/data/class-recordings
|
||||
```
|
||||
|
||||
Create class subfolders:
|
||||
```bash
|
||||
mkdir -p /mnt/data/class-recordings/CS101-Java
|
||||
mkdir -p /mnt/data/class-recordings/MATH201
|
||||
```
|
||||
|
||||
Then restart n8n and activate the workflow. Any `.wav` dropped into a subfolder triggers automatic transcription + RAG ingest.
|
||||
|
||||
### Trigger RSS digest manually:
|
||||
```bash
|
||||
curl -X POST https://n8n.paccoco.com/webhook/scrape/summarize
|
||||
```
|
||||
|
||||
### Test git webhook:
|
||||
```bash
|
||||
curl -X POST https://n8n.paccoco.com/webhook/git/push \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d '{
|
||||
"ref": "refs/heads/main",
|
||||
"pusher": {"name": "john"},
|
||||
"repository": {"full_name": "john/my-project"},
|
||||
"commits": [
|
||||
{"id": "abc1234", "message": "Fix null pointer in UserService", "author": {"name": "john"}, "added": [], "modified": ["src/UserService.java"], "removed": []}
|
||||
]
|
||||
}'
|
||||
```
|
||||
|
||||
## Customizing RSS Feeds (Workflow 06)
|
||||
|
||||
Edit the "Define RSS Feeds" Code node to add/remove feeds. Default feeds:
|
||||
- Self-Hosted Show
|
||||
- r/selfhosted
|
||||
- r/homelab
|
||||
- Hacker News
|
||||
|
||||
Add any RSS feed URL you want monitored.
|
||||
Reference in New Issue
Block a user