Add n8n workflows 01-08 (with upgrades), 14-16; planning docs; .gitignore
This commit is contained in:
Paccoco
29
.gitignore
vendored
29
.gitignore
vendored
@@ -1,16 +1,27 @@
|
||||
# Never commit real secrets
|
||||
# Environment files
|
||||
.env
|
||||
.env.*
|
||||
!.env.example
|
||||
|
||||
# Postgres init SQL contains hardcoded passwords (Option A decision)
|
||||
# Copy from databases/initdb/01-create-databases.sql.example and fill in real values
|
||||
databases/initdb/01-create-databases.sql
|
||||
# Backup files
|
||||
*.bak
|
||||
*.bak.*
|
||||
documents/*.bak*
|
||||
|
||||
# OS noise
|
||||
.DS_Store
|
||||
Thumbs.db
|
||||
# Python
|
||||
__pycache__/
|
||||
*.pyc
|
||||
*.pyo
|
||||
|
||||
# Editor noise
|
||||
# Temp scripts
|
||||
update-homelab-docs.py
|
||||
|
||||
# Editor
|
||||
.vscode/
|
||||
.idea/
|
||||
*.swp
|
||||
*.swo
|
||||
*~
|
||||
|
||||
# OS
|
||||
.DS_Store
|
||||
Thumbs.db
|
||||
|
||||
65
DEPLOYMENT_HANDOFF.md
Normal file
65
DEPLOYMENT_HANDOFF.md
Normal file
@@ -0,0 +1,65 @@
|
||||
# Deployment Handoff — Resume Here
|
||||
|
||||
## Where We Left Off
|
||||
|
||||
Phase 2 (AI stack) is fully deployed and verified on PlausibleDeniability (10.5.1.6). We were about to verify Gotify (Phase 1) and then deploy n8n (Phase 3).
|
||||
|
||||
## What's Done
|
||||
|
||||
### Phase 2 — AI Stack (COMPLETE)
|
||||
All 6 containers running and verified on PD (10.5.1.6):
|
||||
- **Ollama:** port 11434 — qwen2.5:14b loaded
|
||||
- **OpenWebUI:** port 8282 (→8080)
|
||||
- **Qdrant:** port 6333 (HTTP), 6334 (gRPC)
|
||||
- **LiteLLM:** port 4000 — auth working, inference verified (medium tier → qwen2.5:14b)
|
||||
- **Reranker (TEI):** port 8787 (→80) — bge-reranker-v2-m3 via safetensors
|
||||
- **Whisper:** port 8786 (→8000)
|
||||
|
||||
### Key Fixes Applied (reference for future troubleshooting)
|
||||
1. **TrueNAS Scale Docker quirk:** `curl localhost:PORT` returns 000. Always use `curl 10.5.1.6:PORT`.
|
||||
2. **Reranker image:** Use `cpu-latest` not `cpu-1.5` (hf-hub bug).
|
||||
3. **LiteLLM config:** Remove `database_url` (newer versions try Prisma/PostgreSQL). Set `master_key: "os.environ/LITELLM_MASTER_KEY"` to read from env var.
|
||||
|
||||
### Phase 1 — Gotify (DEPLOYED, NOT VERIFIED)
|
||||
Gotify was deployed previously but hasn't been verified this session.
|
||||
|
||||
## What's Next (in order)
|
||||
|
||||
### 1. Verify Gotify
|
||||
Run these on PD and confirm output:
|
||||
```bash
|
||||
docker ps --filter name=gotify --format "table {{.Names}}\t{{.Image}}\t{{.Status}}\t{{.Ports}}"
|
||||
docker logs gotify --tail 20
|
||||
docker inspect gotify --format '{{json .Mounts}}' | python3 -m json.tool
|
||||
docker inspect gotify --format '{{json .NetworkSettings.Networks}}' | python3 -m json.tool
|
||||
curl -sw "\n%{http_code}" http://10.5.1.6:8484/health
|
||||
ls -la /mnt/docker-ssd/docker/compose/automation/
|
||||
cat /mnt/docker-ssd/docker/compose/automation/docker-compose.yaml
|
||||
```
|
||||
|
||||
### 2. Deploy n8n (Phase 3)
|
||||
Per HOMELAB_BUILDOUT_PLAN.md Phase 3:
|
||||
- Create postgres user/db for n8n on shared-postgres
|
||||
- Scaffold `/mnt/tank/docker/appdata/n8n` (chown 1000:1000)
|
||||
- Update automation compose to add n8n alongside Gotify
|
||||
- Generate secrets for .env (N8N_DB_PASS, N8N_ENCRYPTION_KEY)
|
||||
- Deploy and verify (DB migration, network connectivity, UI access)
|
||||
|
||||
### 3. Pending Config (do whenever)
|
||||
- OpenWebUI → add LiteLLM as OpenAI connection (`http://litellm:4000/v1`)
|
||||
- OpenWebUI → optionally set RAG vector DB to Qdrant (`http://qdrant:6333`)
|
||||
|
||||
## Network Reference
|
||||
| Network | Purpose |
|
||||
|---------|---------|
|
||||
| ai-services | Cross-stack: AI containers ↔ n8n, future services |
|
||||
| ix-databases_shared-databases | Shared postgres/redis access |
|
||||
| pangolin | Reverse proxy / external exposure |
|
||||
|
||||
## Machine IPs
|
||||
- **PlausibleDeniability (PD):** 10.5.1.6 — TrueNAS Scale, RTX 2080 Ti
|
||||
- **ROCINANTE:** 10.5.1.112 — RTX 4090
|
||||
- **N.O.M.A.D.:** 10.5.1.16 — Ubuntu, GTX 1080
|
||||
|
||||
## LiteLLM .env Secrets (on PD at /mnt/docker-ssd/docker/compose/ai/.env)
|
||||
Already configured — do not regenerate. Master key is in the .env file as LITELLM_MASTER_KEY.
|
||||
2078
HOMELAB_BUILDOUT_PLAN.md
Normal file
2078
HOMELAB_BUILDOUT_PLAN.md
Normal file
File diff suppressed because it is too large
Load Diff
91
docs/planning/DEPLOY_KIMA_HUB.md
Normal file
91
docs/planning/DEPLOY_KIMA_HUB.md
Normal file
@@ -0,0 +1,91 @@
|
||||
# Kima-Hub Deployment Plan
|
||||
|
||||
**Host:** PlausibleDeniability (PD)
|
||||
**Port:** 3333
|
||||
**Stack dir:** `/mnt/docker-ssd/docker/compose/media-extra/`
|
||||
**Status:** Pending
|
||||
|
||||
## What It Is
|
||||
|
||||
Kima-Hub is a self-hosted music streaming platform — think personal Spotify. Point it at your music library and it handles cataloging, artist discovery, AI-powered playlists, podcast subscriptions, and seamless integration with Lidarr and Audiobookshelf.
|
||||
|
||||
## Pre-Deploy Checklist
|
||||
|
||||
- [ ] Confirm music library path on Unraid NFS (`/mnt/unraid/data/media/music` or similar)
|
||||
- [ ] Create appdata directory: `sudo mkdir -p /mnt/tank/docker/appdata/kima-hub`
|
||||
- [ ] Create stack directory: `sudo mkdir -p /mnt/docker-ssd/docker/compose/media-extra`
|
||||
|
||||
## docker-compose.yaml
|
||||
|
||||
```yaml
|
||||
services:
|
||||
kima-hub:
|
||||
image: chevron7locked/kima:latest
|
||||
container_name: kima-hub
|
||||
restart: unless-stopped
|
||||
ports:
|
||||
- "3333:3030"
|
||||
volumes:
|
||||
- /mnt/tank/docker/appdata/kima-hub:/data
|
||||
- /mnt/unraid/data/media/music:/music:ro
|
||||
environment:
|
||||
- TZ=${TZ}
|
||||
healthcheck:
|
||||
test: ["CMD-SHELL", "cat /proc/net/tcp6 | grep -q 0BD6 || exit 1"]
|
||||
interval: 30s
|
||||
timeout: 10s
|
||||
retries: 3
|
||||
networks:
|
||||
- pangolin
|
||||
|
||||
networks:
|
||||
pangolin:
|
||||
external: true
|
||||
```
|
||||
|
||||
> **Note:** Internal port is `3030`, mapped to `3333` on the host. Hex `0BD6` = port 3030 for healthcheck.
|
||||
> **Music path:** Adjust `/mnt/unraid/data/media/music` to match your actual Unraid music library path.
|
||||
> **Read-only mount:** Music library is mounted `:ro` — Kima-Hub should not need write access to source files.
|
||||
|
||||
## .env.example
|
||||
|
||||
```env
|
||||
TZ=America/New_York
|
||||
```
|
||||
|
||||
## Deploy
|
||||
|
||||
```bash
|
||||
cd /mnt/docker-ssd/docker/compose/media-extra
|
||||
sudo docker compose --env-file .env config # validate
|
||||
sudo docker compose --env-file .env up -d
|
||||
sudo docker logs kima-hub --tail 30
|
||||
```
|
||||
|
||||
## Post-Deploy
|
||||
|
||||
1. Open `http://10.5.1.6:3333` and create your account
|
||||
2. Point Kima at your music directory (`/music`)
|
||||
3. In Lidarr: Settings → Connect → add Kima-Hub webhook
|
||||
4. In Audiobookshelf: verify Kima can reach it at `http://10.5.1.6:13358`
|
||||
5. Add Pangolin resource: `kima.paccoco.com` → port 3333
|
||||
|
||||
## Pangolin Resource
|
||||
|
||||
Add in Pangolin dashboard:
|
||||
- **Subdomain:** `kima.paccoco.com`
|
||||
- **Target:** `http://10.5.1.6:3333`
|
||||
- **Auth:** Enable (personal use only)
|
||||
|
||||
## Storage Decision
|
||||
|
||||
| Path | Tier | Reason |
|
||||
|------|------|--------|
|
||||
| `/mnt/tank/docker/appdata/kima-hub` | tank | Config, DB, playlists — not write-heavy |
|
||||
| `/mnt/unraid/data/media/music` | Unraid NFS (read-only) | Source library stays on Serenity |
|
||||
|
||||
## Notes
|
||||
|
||||
- NFS mount must be up before starting (`sudo bash /mnt/tank/docker/scripts/mount-unraid-nfs.sh`)
|
||||
- If you transcode on the fly, CPU load on PD will spike; Kima-Hub is CPU-bound for transcoding
|
||||
- Kima-Hub has no GPU acceleration — pure CPU transcoding
|
||||
527
docs/planning/DEPLOY_PIHOLE.md
Normal file
527
docs/planning/DEPLOY_PIHOLE.md
Normal file
@@ -0,0 +1,527 @@
|
||||
# Pi-hole Deployment Plan
|
||||
|
||||
**Status:** Pending
|
||||
**Architecture:** 3-node HA cluster with Keepalived VIP, Unbound recursive DNS, Nebula Sync
|
||||
|
||||
---
|
||||
|
||||
## Architecture Overview
|
||||
|
||||
```
|
||||
[All LAN clients]
|
||||
↓ DNS: 10.5.1.53 (Virtual IP — floats to whichever node is MASTER)
|
||||
↓
|
||||
┌─────┴──────────────────────────────────┐
|
||||
│ Keepalived VIP │
|
||||
│ 10.5.1.53/24 │
|
||||
└──────┬──────────────┬──────────────────┘
|
||||
│ │ │
|
||||
PD (MASTER) Serenity (BACKUP1) RPi4 (BACKUP2)
|
||||
10.5.1.6 10.5.1.5 10.5.1.X
|
||||
priority 100 priority 90 priority 80
|
||||
Pi-hole :53 Pi-hole :53 Pi-hole :53
|
||||
Unbound :5335 Unbound :5335 Unbound :5335
|
||||
│ │ │
|
||||
└──────────────┴──────────────────┘
|
||||
↓
|
||||
[Root DNS servers]
|
||||
(recursive — no third-party upstream)
|
||||
```
|
||||
|
||||
- **Keepalived** floats the VIP `10.5.1.53` to whichever node is alive with the highest priority
|
||||
- **Unbound** on each node acts as the local recursive resolver (replaces Cloudflare/Quad9)
|
||||
- **Pi-hole** on each node forwards upstream queries to its local Unbound (`127.0.0.1#5335`)
|
||||
- **Nebula Sync** (running on PD) syncs all Pi-hole configs from primary → replicas daily
|
||||
|
||||
All nodes use `network_mode: host` for Pi-hole, Unbound, and Keepalived — this is required for Keepalived to manipulate the VIP and for Pi-hole to bind cleanly to port 53.
|
||||
|
||||
---
|
||||
|
||||
## Pre-Deploy: Network Planning
|
||||
|
||||
1. **Pick a free static IP for the VIP** — `10.5.1.53` is recommended (memorable for DNS). Reserve it in UniFi so DHCP never assigns it.
|
||||
2. **Find the LAN interface name** on each node:
|
||||
```bash
|
||||
# PD / RPi4:
|
||||
ip link show
|
||||
# Look for the interface with 10.5.1.x address — likely eth0, eno1, enp3s0, etc.
|
||||
```
|
||||
3. **Note RPi4 IP** — fill in `10.5.1.X` throughout this doc once known. Reserve it as static in UniFi.
|
||||
4. **Verify port 53 is free on PD:**
|
||||
```bash
|
||||
sudo ss -tulnp | grep ':53'
|
||||
# Should show only 127.0.0.1:53 (TrueNAS loopback resolver).
|
||||
# If it shows 0.0.0.0:53, see Gotchas section.
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Unbound Config (shared across all nodes)
|
||||
|
||||
Each node runs Unbound as a Docker container on the host network, listening on `127.0.0.1:5335`.
|
||||
|
||||
**Save this as `unbound.conf` in each node's appdata path** (see per-node sections below):
|
||||
|
||||
```ini
|
||||
server:
|
||||
verbosity: 1
|
||||
interface: 127.0.0.1
|
||||
port: 5335
|
||||
do-ip4: yes
|
||||
do-udp: yes
|
||||
do-tcp: yes
|
||||
do-ip6: no
|
||||
|
||||
# DNSSEC
|
||||
auto-trust-anchor-file: "/opt/unbound/etc/unbound/var/root.key"
|
||||
val-permissive-mode: no
|
||||
|
||||
# Performance
|
||||
num-threads: 2
|
||||
cache-min-ttl: 300
|
||||
cache-max-ttl: 86400
|
||||
prefetch: yes
|
||||
prefetch-key: yes
|
||||
minimal-responses: yes
|
||||
|
||||
# Privacy
|
||||
qname-minimisation: yes
|
||||
hide-identity: yes
|
||||
hide-version: yes
|
||||
|
||||
# Access control
|
||||
access-control: 127.0.0.1/32 allow
|
||||
access-control: 0.0.0.0/0 refuse
|
||||
|
||||
# Root hints — download updated copy periodically
|
||||
root-hints: "/opt/unbound/etc/unbound/var/root.hints"
|
||||
```
|
||||
|
||||
**Download root hints** (run once on each node, into the appdata path):
|
||||
```bash
|
||||
curl -o /path/to/appdata/unbound/var/root.hints https://www.internic.net/domain/named.cache
|
||||
mkdir -p /path/to/appdata/unbound/var
|
||||
# Unbound will auto-manage root.key for DNSSEC
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Node 1: PlausibleDeniability (MASTER)
|
||||
|
||||
**Stack dir:** `/mnt/docker-ssd/docker/compose/pihole/`
|
||||
**Web UI port:** `8953`
|
||||
|
||||
### Pre-Deploy
|
||||
|
||||
```bash
|
||||
sudo mkdir -p /mnt/tank/docker/appdata/pihole-primary/etc-pihole
|
||||
sudo mkdir -p /mnt/tank/docker/appdata/unbound-primary/var
|
||||
sudo mkdir -p /mnt/docker-ssd/docker/compose/pihole
|
||||
# Download root hints
|
||||
sudo curl -o /mnt/tank/docker/appdata/unbound-primary/var/root.hints https://www.internic.net/domain/named.cache
|
||||
```
|
||||
|
||||
### docker-compose.yaml
|
||||
|
||||
```yaml
|
||||
services:
|
||||
unbound:
|
||||
image: mvance/unbound:latest
|
||||
container_name: unbound
|
||||
network_mode: host
|
||||
restart: unless-stopped
|
||||
volumes:
|
||||
- /mnt/tank/docker/appdata/unbound-primary:/opt/unbound/etc/unbound/
|
||||
healthcheck:
|
||||
test: ["CMD-SHELL", "drill @127.0.0.1 -p 5335 cloudflare.com || exit 1"]
|
||||
interval: 30s
|
||||
timeout: 10s
|
||||
retries: 3
|
||||
|
||||
pihole:
|
||||
image: pihole/pihole:latest
|
||||
container_name: pihole
|
||||
network_mode: host
|
||||
restart: unless-stopped
|
||||
depends_on:
|
||||
unbound:
|
||||
condition: service_healthy
|
||||
environment:
|
||||
TZ: ${TZ}
|
||||
FTLCONF_webserver_api_password: ${PIHOLE_PASSWORD}
|
||||
FTLCONF_dns_listeningMode: "LOCAL"
|
||||
FTLCONF_dns_upstreams: "127.0.0.1#5335"
|
||||
FTLCONF_webserver_port: "8953"
|
||||
volumes:
|
||||
- /mnt/tank/docker/appdata/pihole-primary/etc-pihole:/etc/pihole
|
||||
cap_add:
|
||||
- SYS_NICE
|
||||
|
||||
keepalived:
|
||||
image: osixia/keepalived:2.0.20
|
||||
container_name: keepalived
|
||||
network_mode: host
|
||||
restart: unless-stopped
|
||||
cap_add:
|
||||
- NET_ADMIN
|
||||
- NET_BROADCAST
|
||||
- NET_RAW
|
||||
volumes:
|
||||
- ./keepalived.conf:/container/environment/01-custom/keepalived.conf
|
||||
environment:
|
||||
KEEPALIVED_VIRTUAL_IPS: "#PYTHON2BASH:['10.5.1.53/24']"
|
||||
```
|
||||
|
||||
### keepalived.conf (PD — MASTER)
|
||||
|
||||
```
|
||||
global_defs {
|
||||
router_id pihole_pd
|
||||
}
|
||||
|
||||
vrrp_instance VI_DNS {
|
||||
state MASTER
|
||||
interface ${LAN_INTERFACE}
|
||||
virtual_router_id 53
|
||||
priority 100
|
||||
advert_int 1
|
||||
nopreempt
|
||||
|
||||
authentication {
|
||||
auth_type PASS
|
||||
auth_pass ${KEEPALIVED_PASSWORD}
|
||||
}
|
||||
|
||||
virtual_ipaddress {
|
||||
10.5.1.53/24
|
||||
}
|
||||
}
|
||||
```
|
||||
> Replace `${LAN_INTERFACE}` with your actual interface (e.g. `eno1`). Replace `${KEEPALIVED_PASSWORD}` with a shared secret used on all three nodes.
|
||||
|
||||
### .env.example
|
||||
|
||||
```env
|
||||
TZ=America/New_York
|
||||
PIHOLE_PASSWORD=CHANGE_ME
|
||||
KEEPALIVED_PASSWORD=CHANGE_ME
|
||||
LAN_INTERFACE=eno1
|
||||
```
|
||||
|
||||
### Deploy
|
||||
|
||||
```bash
|
||||
cd /mnt/docker-ssd/docker/compose/pihole
|
||||
# Copy unbound.conf to appdata
|
||||
sudo cp unbound.conf /mnt/tank/docker/appdata/unbound-primary/
|
||||
sudo docker compose --env-file .env config
|
||||
sudo docker compose --env-file .env up -d
|
||||
sudo docker logs pihole --tail 20
|
||||
sudo docker logs unbound --tail 20
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Node 2: Serenity (BACKUP1)
|
||||
|
||||
Serenity runs Unraid. Deploy via Docker Compose in Unraid's **User Scripts** or the **Compose Manager** plugin.
|
||||
|
||||
**Appdata:** `/mnt/user/appdata/`
|
||||
**Web UI port:** `8954`
|
||||
|
||||
### docker-compose.yaml
|
||||
|
||||
Same structure as PD but with different ports/paths and priority:
|
||||
|
||||
```yaml
|
||||
services:
|
||||
unbound:
|
||||
image: mvance/unbound:latest
|
||||
container_name: unbound-pihole
|
||||
network_mode: host
|
||||
restart: unless-stopped
|
||||
volumes:
|
||||
- /mnt/user/appdata/unbound-pihole:/opt/unbound/etc/unbound/
|
||||
healthcheck:
|
||||
test: ["CMD-SHELL", "drill @127.0.0.1 -p 5335 cloudflare.com || exit 1"]
|
||||
interval: 30s
|
||||
timeout: 10s
|
||||
retries: 3
|
||||
|
||||
pihole:
|
||||
image: pihole/pihole:latest
|
||||
container_name: pihole-secondary
|
||||
network_mode: host
|
||||
restart: unless-stopped
|
||||
depends_on:
|
||||
unbound-pihole:
|
||||
condition: service_healthy
|
||||
environment:
|
||||
TZ: ${TZ}
|
||||
FTLCONF_webserver_api_password: ${PIHOLE_PASSWORD}
|
||||
FTLCONF_dns_listeningMode: "LOCAL"
|
||||
FTLCONF_dns_upstreams: "127.0.0.1#5335"
|
||||
FTLCONF_webserver_port: "8954"
|
||||
volumes:
|
||||
- /mnt/user/appdata/pihole-secondary/etc-pihole:/etc/pihole
|
||||
cap_add:
|
||||
- SYS_NICE
|
||||
|
||||
keepalived:
|
||||
image: osixia/keepalived:2.0.20
|
||||
container_name: keepalived-pihole
|
||||
network_mode: host
|
||||
restart: unless-stopped
|
||||
cap_add:
|
||||
- NET_ADMIN
|
||||
- NET_BROADCAST
|
||||
- NET_RAW
|
||||
volumes:
|
||||
- ./keepalived.conf:/container/environment/01-custom/keepalived.conf
|
||||
```
|
||||
|
||||
### keepalived.conf (Serenity — BACKUP1)
|
||||
|
||||
```
|
||||
global_defs {
|
||||
router_id pihole_serenity
|
||||
}
|
||||
|
||||
vrrp_instance VI_DNS {
|
||||
state BACKUP
|
||||
interface eth0
|
||||
virtual_router_id 53
|
||||
priority 90
|
||||
advert_int 1
|
||||
|
||||
authentication {
|
||||
auth_type PASS
|
||||
auth_pass SAME_PASSWORD_AS_PRIMARY
|
||||
}
|
||||
|
||||
virtual_ipaddress {
|
||||
10.5.1.53/24
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Node 3: Raspberry Pi 4 (BACKUP2)
|
||||
|
||||
### Install Docker (if not already installed)
|
||||
|
||||
```bash
|
||||
curl -fsSL https://get.docker.com | sh
|
||||
sudo usermod -aG docker $USER
|
||||
```
|
||||
|
||||
### Pre-Deploy
|
||||
|
||||
```bash
|
||||
mkdir -p ~/pihole/etc-pihole
|
||||
mkdir -p ~/pihole/unbound/var
|
||||
curl -o ~/pihole/unbound/var/root.hints https://www.internic.net/domain/named.cache
|
||||
sudo apt install keepalived -y
|
||||
```
|
||||
|
||||
### docker-compose.yaml
|
||||
|
||||
```yaml
|
||||
services:
|
||||
unbound:
|
||||
image: mvance/unbound:latest
|
||||
container_name: unbound
|
||||
network_mode: host
|
||||
restart: unless-stopped
|
||||
volumes:
|
||||
- ~/pihole/unbound:/opt/unbound/etc/unbound/
|
||||
healthcheck:
|
||||
test: ["CMD-SHELL", "drill @127.0.0.1 -p 5335 cloudflare.com || exit 1"]
|
||||
interval: 30s
|
||||
timeout: 10s
|
||||
retries: 3
|
||||
|
||||
pihole:
|
||||
image: pihole/pihole:latest
|
||||
container_name: pihole
|
||||
network_mode: host
|
||||
restart: unless-stopped
|
||||
depends_on:
|
||||
unbound:
|
||||
condition: service_healthy
|
||||
environment:
|
||||
TZ: ${TZ}
|
||||
FTLCONF_webserver_api_password: ${PIHOLE_PASSWORD}
|
||||
FTLCONF_dns_listeningMode: "LOCAL"
|
||||
FTLCONF_dns_upstreams: "127.0.0.1#5335"
|
||||
FTLCONF_webserver_port: "8955"
|
||||
volumes:
|
||||
- ~/pihole/etc-pihole:/etc/pihole
|
||||
cap_add:
|
||||
- SYS_NICE
|
||||
```
|
||||
|
||||
### keepalived (native on RPi4)
|
||||
|
||||
```bash
|
||||
sudo nano /etc/keepalived/keepalived.conf
|
||||
```
|
||||
|
||||
```
|
||||
global_defs {
|
||||
router_id pihole_rpi4
|
||||
}
|
||||
|
||||
vrrp_instance VI_DNS {
|
||||
state BACKUP
|
||||
interface eth0
|
||||
virtual_router_id 53
|
||||
priority 80
|
||||
advert_int 1
|
||||
|
||||
authentication {
|
||||
auth_type PASS
|
||||
auth_pass SAME_PASSWORD_AS_PRIMARY
|
||||
}
|
||||
|
||||
virtual_ipaddress {
|
||||
10.5.1.53/24
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
```bash
|
||||
sudo systemctl enable keepalived
|
||||
sudo systemctl start keepalived
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Nebula Sync (on PD)
|
||||
|
||||
[Nebula Sync](https://github.com/lovelaze/nebula-sync) syncs Pi-hole v6 configuration (gravity, blocklists, allowlists, custom DNS) from the primary to both replicas.
|
||||
|
||||
**Add to PD's pihole docker-compose.yaml:**
|
||||
|
||||
```yaml
|
||||
nebula-sync:
|
||||
image: lovelaze/nebula-sync:latest
|
||||
container_name: nebula-sync
|
||||
restart: unless-stopped
|
||||
environment:
|
||||
PRIMARY: "http://127.0.0.1:8953|${PIHOLE_PASSWORD}"
|
||||
REPLICAS: "http://10.5.1.5:8954|${PIHOLE_PASSWORD_SERENITY},http://10.5.1.X:8955|${PIHOLE_PASSWORD_RPI}"
|
||||
RUN_GRAVITY: "true"
|
||||
FULL_SYNC: "false"
|
||||
CRON: "0 3 * * *"
|
||||
networks:
|
||||
- default
|
||||
```
|
||||
|
||||
> - `PRIMARY` — your PD Pi-hole (note: uses `127.0.0.1` since nebula-sync is on the same host; port 8953 is the web UI, also the API port in Pi-hole v6)
|
||||
> - `REPLICAS` — comma-separated list of secondary instances
|
||||
> - `CRON` — syncs daily at 3 AM; force a manual sync with `docker exec nebula-sync nebula-sync run`
|
||||
> - Each Pi-hole instance can (and should) have a **different** password — nebula-sync handles auth per-instance
|
||||
> - Add `PIHOLE_PASSWORD_SERENITY` and `PIHOLE_PASSWORD_RPI` to your `.env` file
|
||||
|
||||
### What Nebula Sync copies
|
||||
- Gravity database (blocklists, allowlists, group assignments)
|
||||
- Custom DNS/CNAME records
|
||||
- Client groups
|
||||
- Does **not** sync per-instance settings (web password, interface binding — these stay local)
|
||||
|
||||
---
|
||||
|
||||
## Router Configuration (UniFi)
|
||||
|
||||
Point **only the VIP** as DNS:
|
||||
|
||||
1. UniFi Network → Settings → Networks → **LAN** → DHCP Name Server: **Manual**
|
||||
2. **DNS Server 1:** `10.5.1.53` (VIP — floats between nodes)
|
||||
3. **DNS Server 2:** *(leave blank or set a public fallback like `1.1.1.1` only if you want internet DNS as last resort)*
|
||||
4. Save and renew DHCP leases
|
||||
|
||||
> **Why single VIP instead of listing all three IPs?**
|
||||
> Clients round-robin across all listed DNS servers — so some queries bypass Pi-hole entirely. With Keepalived the VIP always points to exactly one live Pi-hole node, ensuring 100% of queries are filtered.
|
||||
|
||||
---
|
||||
|
||||
## Post-Deploy
|
||||
|
||||
### Verify VIP is working
|
||||
|
||||
```bash
|
||||
# Check which node holds the VIP
|
||||
ip addr show | grep 10.5.1.53 # Run on each node — only MASTER shows it
|
||||
|
||||
# Test DNS through VIP
|
||||
nslookup google.com 10.5.1.53
|
||||
nslookup doubleclick.net 10.5.1.53 # Should return 0.0.0.0 (blocked)
|
||||
```
|
||||
|
||||
### Test failover
|
||||
|
||||
```bash
|
||||
# On PD, bring down keepalived
|
||||
sudo docker stop keepalived
|
||||
# Wait ~3 seconds, then check from another machine:
|
||||
nslookup google.com 10.5.1.53 # Should still work — Serenity took over VIP
|
||||
# Restore
|
||||
sudo docker start keepalived
|
||||
```
|
||||
|
||||
### Add blocklists (on primary — Nebula Sync propagates to replicas)
|
||||
|
||||
| List | URL |
|
||||
|------|-----|
|
||||
| Steven Black (unified) | `https://raw.githubusercontent.com/StevenBlack/hosts/master/hosts` |
|
||||
| OISD Basic | `https://basic.oisd.nl/` |
|
||||
| Hagezi Pro | `https://raw.githubusercontent.com/hagezi/dns-blocklists/main/adblock/pro.txt` |
|
||||
|
||||
After adding: **Tools → Update Gravity** on PD, then run Nebula Sync to push to replicas.
|
||||
|
||||
### Local DNS records
|
||||
|
||||
Add in Pi-hole (primary) → Local DNS → DNS Records:
|
||||
|
||||
| Domain | IP |
|
||||
|--------|----|
|
||||
| pd.lan | 10.5.1.6 |
|
||||
| serenity.lan | 10.5.1.5 |
|
||||
| nomad.lan | 10.5.1.16 |
|
||||
| rocinante.lan | 10.5.1.112 |
|
||||
| rpi4.lan | 10.5.1.X |
|
||||
|
||||
Nebula Sync will propagate these to all replicas.
|
||||
|
||||
### Pangolin for admin UIs (optional)
|
||||
|
||||
| Subdomain | Target |
|
||||
|-----------|--------|
|
||||
| `pihole.paccoco.com` | `http://10.5.1.6:8953` (primary) |
|
||||
| `pihole2.paccoco.com` | `http://10.5.1.5:8954` (Serenity) |
|
||||
|
||||
Enable Pangolin auth — Pi-hole admin should not be publicly accessible.
|
||||
|
||||
---
|
||||
|
||||
## Storage Decisions
|
||||
|
||||
| Node | Path | Tier | Reason |
|
||||
|------|------|------|--------|
|
||||
| PD | `/mnt/tank/docker/appdata/pihole-primary` | tank | Low write frequency |
|
||||
| PD | `/mnt/tank/docker/appdata/unbound-primary` | tank | Config + root hints only |
|
||||
| Serenity | `/mnt/user/appdata/pihole-secondary` | Unraid array | Low write frequency |
|
||||
| RPi4 | `~/pihole/` | SD card / SSD | Dedicated Pi, no other concerns |
|
||||
|
||||
---
|
||||
|
||||
## Known Gotchas
|
||||
|
||||
- **Port 53 conflict on PD:** TrueNAS binds to `127.0.0.1:53`. With `network_mode: host`, Pi-hole will try to bind to all interfaces. Set `FTLCONF_dns_listeningMode: LOCAL` so it only listens on the actual LAN interface, not loopback.
|
||||
- **keepalived interface name:** Must match the actual NIC name on each host. Get it with `ip link show`.
|
||||
- **`virtual_router_id` must be unique** per VRRP group on your LAN. `53` is a good mnemonic. Confirm no other keepalived instance on your LAN uses the same ID.
|
||||
- **Pi-hole v6 API port:** In v6, the web UI and API both run on the `FTLCONF_webserver_port`. Nebula Sync and the web browser use the same port.
|
||||
- **nopreempt on MASTER:** The `nopreempt` option in PD's keepalived.conf prevents PD from immediately reclaiming the VIP when it comes back up after a failure. Remove it if you want PD to always retake MASTER on recovery.
|
||||
- **Unbound cold start:** Unbound takes a few seconds to start and validate DNSSEC. The `depends_on: condition: service_healthy` in the Pi-hole compose ensures Pi-hole waits for Unbound before starting.
|
||||
186
docs/planning/N8N_WORKFLOW_IDEAS.md
Normal file
186
docs/planning/N8N_WORKFLOW_IDEAS.md
Normal file
@@ -0,0 +1,186 @@
|
||||
# n8n Workflow Ideas & Upgrade Proposals
|
||||
|
||||
Generated: 2026-05-09
|
||||
|
||||
---
|
||||
|
||||
## Upgrades to Existing Workflows
|
||||
|
||||
### 01 — Grafana Alert → AI → Gotify
|
||||
**Current state:** Webhook → parse → LiteLLM summarize → Gotify
|
||||
**Suggested upgrades:**
|
||||
- **Alert deduplication:** Store last-seen alert fingerprint in Postgres. If the same alert fires again within 30 min, suppress the Gotify push (you're already awake, you've seen it).
|
||||
- **Recovery notifications:** Grafana sends both `firing` and `resolved` states. Add a branch so resolved alerts get a `✅ RESOLVED` message with a distinct priority (1 = low).
|
||||
- **Auto-acknowledge low-priority alerts:** If priority < 3 and it's between midnight and 7am, skip Gotify and log to Postgres only.
|
||||
|
||||
---
|
||||
|
||||
### 04 — Whisper Audio Transcription
|
||||
**Current state:** Returns transcript inline, no Gotify notification
|
||||
**Suggested upgrades:**
|
||||
- **Add completion notification:** When a long transcription finishes, push to Gotify (Whisper Transcriptions app) so you know it's done without polling.
|
||||
- **Save transcript to Paperless:** After transcription, POST the text to Paperless-NGX as a new document with auto-tagging (`transcription`, `audio`, date).
|
||||
- **Save to Qdrant:** After transcription + summary, auto-ingest into RAG so all your audio content is searchable.
|
||||
|
||||
---
|
||||
|
||||
### 05 — Paperless → RAG Ingest
|
||||
**Current state:** Webhook → fetch → chunk → embed → Qdrant → Gotify
|
||||
**Suggested upgrades:**
|
||||
- **Retry on failure:** If Qdrant upsert fails, retry up to 3 times with exponential backoff before erroring out.
|
||||
- **Collection routing:** Add metadata tagging so documents from different Paperless tags go into different Qdrant collections (e.g., `knowledge_base` vs `receipts` vs `medical`).
|
||||
|
||||
---
|
||||
|
||||
### 08 — Class Recording → Transcribe → RAG
|
||||
**Current state:** Webhook → Whisper → AI notes → RAG → Gotify
|
||||
**Suggested upgrades:**
|
||||
- **Export notes to Paperless:** After extracting key concepts, create a Paperless document for the class notes (title = class + lecture, tags = class name, `lecture-notes`).
|
||||
- **Weekly class digest:** Scheduled workflow that queries RAG for all chunks from the past week, grouped by class, and sends a consolidated study summary to Gotify every Sunday night.
|
||||
- **Quiz generation:** Add an optional webhook endpoint that takes a class name + RAG query and generates 5 practice questions from the lecture material via LiteLLM.
|
||||
|
||||
---
|
||||
|
||||
## New Workflow Proposals
|
||||
|
||||
### 09 — Immich → AI Event Tagger
|
||||
**Trigger:** Immich post-upload webhook (or scheduled poll via Immich API)
|
||||
**What it does:**
|
||||
1. Fetch newly uploaded photos from Immich API
|
||||
2. Send image to a vision-capable model (gemma3:27b on Rocinante via LiteLLM)
|
||||
3. AI generates descriptive tags, detects events (birthday, trip, holiday), extracts visible text
|
||||
4. Write tags back to Immich via API
|
||||
5. Notify via Gotify (Immich app)
|
||||
|
||||
**Why:** Immich's built-in ML does faces, not semantic tagging. This adds searchable event context automatically.
|
||||
|
||||
---
|
||||
|
||||
### 10 — Home Assistant → Morning Briefing
|
||||
**Trigger:** Schedule (e.g. 7:00 AM daily)
|
||||
**What it does:**
|
||||
1. Fetch current weather from Home Assistant or weather API
|
||||
2. Query Grafana/Prometheus for overnight anomalies (high CPU, disk warnings)
|
||||
3. Check Pi-hole stats (queries blocked, top blocked domain)
|
||||
4. Check Paperless for any unread/untagged documents
|
||||
5. Summarize via LiteLLM into a 5-bullet morning briefing
|
||||
6. Push to Gotify (Morning Briefing app) and optionally to Home Assistant notification
|
||||
|
||||
**Why:** Single daily digest so you know what happened overnight without checking 4 different dashboards.
|
||||
|
||||
---
|
||||
|
||||
### 11 — Uptime Kuma → Smart Alert Escalation
|
||||
**Trigger:** Uptime Kuma webhook on status change
|
||||
**What it does:**
|
||||
1. Receive down/recovery event from Uptime Kuma
|
||||
2. Check Prometheus to confirm actual service health (avoids false positives)
|
||||
3. If confirmed down: check how long it's been down
|
||||
- < 2 min: log only (transient blip)
|
||||
- 2–10 min: Gotify push (medium priority)
|
||||
- > 10 min: Gotify push (high priority) + attempt auto-restart via `docker restart` API call to PD
|
||||
4. On recovery: send resolution notification with total downtime
|
||||
|
||||
**Why:** Uptime Kuma already sends webhooks but has no intelligence. This adds confirmation, suppression of transient blips, and auto-remediation for stuck containers.
|
||||
|
||||
---
|
||||
|
||||
### 12 — Pi-hole Stats Digest
|
||||
**Trigger:** Schedule (daily at 8 AM)
|
||||
**What it does:**
|
||||
1. Query Pi-hole API on PD (`http://10.5.1.6:8953/api/stats/summary`)
|
||||
2. Query secondary Pi-hole on Serenity
|
||||
3. LiteLLM summarizes notable blocked domains and compares primary vs secondary query load
|
||||
4. Push digest to Gotify (Pi-hole app) with 24h block count, top blocked domains, and any unusual spikes
|
||||
|
||||
**Why:** Passive DNS visibility — know if something on your network is phoning home unexpectedly.
|
||||
|
||||
---
|
||||
|
||||
### 13 — Lidarr → Kima-Hub New Music Notifier
|
||||
**Trigger:** Lidarr webhook on album download complete
|
||||
**What it does:**
|
||||
1. Receive download event from Lidarr
|
||||
2. Fetch album art and metadata from MusicBrainz or Lidarr API
|
||||
3. Format a rich notification: artist, album, year, genre
|
||||
4. Push to Gotify (Music app) with album art link
|
||||
5. Optionally trigger Kima-Hub library rescan via API
|
||||
|
||||
**Why:** Know when new music lands without watching Lidarr activity logs.
|
||||
|
||||
---
|
||||
|
||||
### 14 — Paperless Inbox Reminder
|
||||
**Trigger:** Schedule (every Monday 9 AM)
|
||||
**What it does:**
|
||||
1. Query Paperless API for documents with no tags or in the inbox
|
||||
2. If count > 0: push reminder to Gotify listing the oldest unprocessed documents
|
||||
3. If inbox is clear: send a ✅ confirmation
|
||||
|
||||
**Why:** It's easy to forget to process scanned documents. This ensures nothing sits in the inbox untagged for weeks.
|
||||
|
||||
---
|
||||
|
||||
### 15 — n8n Self-Health Monitor
|
||||
**Trigger:** Schedule (every 15 min)
|
||||
**What it does:**
|
||||
1. Check n8n's own execution history via API — look for failed executions in the past hour
|
||||
2. If failures found: fetch the error message from n8n API
|
||||
3. Push a "Workflow Failed" notification to Gotify (n8n Health app) with the workflow name and error summary
|
||||
4. Skip if the failure is the same as the last-notified one (dedup via Postgres)
|
||||
|
||||
**Why:** n8n doesn't natively alert you when a workflow silently fails. This closes that blind spot.
|
||||
|
||||
---
|
||||
|
||||
### 16 — NFS Mount Watchdog + Auto-Heal
|
||||
**Trigger:** Schedule (every 5 min)
|
||||
**What it does:**
|
||||
1. SSH into PD (10.5.1.6) and probe each expected NFS mount path with `mountpoint -q`:
|
||||
- `/mnt/unraid/data/media` (Plex, Audiobookshelf)
|
||||
- `/mnt/unraid/data/photos` (Immich)
|
||||
- Any other mounts defined in the mount script
|
||||
2. If all mounts are healthy: silent exit
|
||||
3. If any mount is missing:
|
||||
a. Run the NFS mount script: `sudo bash /mnt/tank/docker/scripts/mount-unraid-nfs.sh`
|
||||
b. Wait 10 seconds, re-probe to confirm mounts came up
|
||||
c. If mounts confirmed: restart only the affected containers:
|
||||
- Media mount missing → `sudo docker restart plex audiobookshelf`
|
||||
- Photos mount missing → `sudo docker restart immich-server immich-microservices immich-machine-learning`
|
||||
d. If mounts still missing after script: escalate to Gotify (high priority — NFS down, manual intervention needed)
|
||||
4. Push result to Gotify (Infra app):
|
||||
- Auto-healed: priority 5, list which mounts were remounted and which containers restarted
|
||||
- Escalation: priority 9, list which paths are still unavailable
|
||||
|
||||
**Implementation notes:**
|
||||
- Uses n8n SSH node connecting to `10.5.1.6` with an SSH credential (key-based auth from n8n container)
|
||||
- SSH node runs a single compound command: `mountpoint -q /mnt/unraid/data/media && echo MEDIA_OK || echo MEDIA_MISSING` etc.
|
||||
- Parse output in a Code node to determine which mounts failed
|
||||
- Use separate SSH nodes for: (1) running the mount script, (2) restarting containers — keeps the flow readable
|
||||
- `continueOnFail: true` on the mount script SSH node so a TrueNAS-unreachable scenario still escalates instead of dying silently
|
||||
- Add a Postgres dedup table (`nfs_heal_log`) so repeated failures within 30 min only send one Gotify push, but every successful auto-heal always notifies
|
||||
|
||||
**Why:** TrueNAS reboots (updates, power blips) drop NFS mounts and crash-loop dependent containers. This closes the gap between "Serenity rebooted at 3 AM" and "you notice Immich is broken at noon."
|
||||
|
||||
---
|
||||
|
||||
## Suggested Gotify App / Channel Structure
|
||||
|
||||
Based on all workflows above, here's the full recommended channel layout:
|
||||
|
||||
| App Name | Workflows |
|
||||
|----------|-----------|
|
||||
| Grafana Alerts | 01 |
|
||||
| Paperless | 03 |
|
||||
| Paperless RAG | 05 |
|
||||
| RSS Feed | 06 |
|
||||
| Git Commits | 07 |
|
||||
| Class Recordings | 08 |
|
||||
| Immich | 09 (new) |
|
||||
| Morning Briefing | 10 (new) |
|
||||
| Uptime / Infra | 11, 16 (new) |
|
||||
| Pi-hole | 12 (new) |
|
||||
| Music | 13 (new) |
|
||||
| Reminders | 14 (new) |
|
||||
| n8n Health | 15 (new) |
|
||||
| Whisper | 04 (after upgrade) |
|
||||
@@ -5,7 +5,9 @@
|
||||
- [ ] Regenerate N.O.M.A.D. Newt secret (was exposed in chat)
|
||||
|
||||
## Infrastructure
|
||||
- [ ] Deploy Pi-hole ×2 (primary on PD + backup on Serenity or Pi4)
|
||||
- [ ] Deploy Pi-hole 3-node HA cluster (PD + Serenity + RPi4) — see docs/planning/DEPLOY_PIHOLE.md
|
||||
- RPi4 IP still TBD — fill in before deploying
|
||||
- [ ] Deploy Kima-Hub — see docs/planning/DEPLOY_KIMA_HUB.md
|
||||
- [ ] Set up off-site backup for vital data (appdata, databases, config repo, photos)
|
||||
- [ ] Remove dead Unraid-Cloudflared-Tunnel container from Serenity
|
||||
- [ ] Serenity malcolm pool capacity planning (heavily utilized)
|
||||
@@ -32,6 +34,14 @@
|
||||
- [ ] Document meshtastic stack fully (see docs/reference/MESHTASTIC.md)
|
||||
- [ ] Document N.O.M.A.D. containers (start_nomad.sh services)
|
||||
|
||||
## n8n Workflows
|
||||
- [ ] Create Gotify apps and n8n credentials: Reminders, n8n Health, Uptime / Infra
|
||||
- [ ] Add env vars to n8n: `PAPERLESS_API_TOKEN`, `PAPERLESS_TAG_TRANSCRIPTION`, `PAPERLESS_TAG_LECTURE_NOTES`, `N8N_API_KEY`
|
||||
- [ ] Set up `PD SSH` credential in n8n for workflow 16 (see n8n-workflows/README.md)
|
||||
- [ ] Add sudoers NOPASSWD rule for n8n watchdog (see n8n-workflows/README.md)
|
||||
- [ ] Import and activate upgraded workflows: 01, 04, 08
|
||||
- [ ] Import and activate new workflows: 14, 15 (16 already active)
|
||||
|
||||
## Completed
|
||||
- [x] Deploy Gotify (Phase 1) — 2026-05-05
|
||||
- [x] Deploy AI stack — LiteLLM, OpenWebUI, Qdrant, Whisper, SearXNG (Phase 2) — 2026-05-05
|
||||
@@ -43,3 +53,7 @@
|
||||
- [x] Fill in port/URL columns in SERVICES_DIRECTORY.md — 2026-05-09
|
||||
- [x] Create ROCINANTE.md server doc — 2026-05-09
|
||||
- [x] Update all docs to reflect deployed state — 2026-05-09
|
||||
- [x] Write n8n workflow upgrade + new workflow plans (01, 04, 08, 14, 15, 16) — 2026-05-09
|
||||
- [x] Write Pi-hole 3-node HA deployment plan — 2026-05-09
|
||||
- [x] Write Kima-Hub deployment plan — 2026-05-09
|
||||
- [x] Create Postgres dedup tables (grafana_alert_dedup, n8n_health_dedup) on PD — 2026-05-09
|
||||
|
||||
@@ -4,6 +4,15 @@ Per-service gotchas that aren't bugs but will bite you if you forget them.
|
||||
|
||||
## PlausibleDeniability
|
||||
|
||||
### n8n workflow 16 (NFS Watchdog)
|
||||
- Runs `sudo docker restart` and `sudo bash .../mount-unraid-nfs.sh` over SSH from n8n container
|
||||
- Requires a NOPASSWD sudoers rule or SSH commands will hang waiting for a password:
|
||||
```
|
||||
truenas_admin ALL=(ALL) NOPASSWD: /usr/bin/docker, /bin/bash /mnt/tank/docker/scripts/mount-unraid-nfs.sh
|
||||
```
|
||||
Add via `sudo visudo -f /etc/sudoers.d/n8n-watchdog`
|
||||
- SSH credential in n8n must use key-based auth (see n8n-workflows/README.md for keygen steps)
|
||||
|
||||
### immich-ml
|
||||
- Healthcheck has no curl/wget — uses `/proc/net/tcp6` pattern
|
||||
- Cache goes to `/mnt/docker-ssd/docker/appdata/immich-ml` (separate from immich-server's appdata)
|
||||
|
||||
0
meshtastic/scripts/write-test
Normal file
0
meshtastic/scripts/write-test
Normal file
363
n8n-workflows/01-grafana-alert-ai-gotify.json
Normal file
363
n8n-workflows/01-grafana-alert-ai-gotify.json
Normal file
@@ -0,0 +1,363 @@
|
||||
{
|
||||
"updatedAt": "2026-05-07T03:32:35.930Z",
|
||||
"createdAt": "2026-05-07T03:32:32.026Z",
|
||||
"id": "GXtkp4xKka90XrXH",
|
||||
"name": "Grafana Alert \u2192 AI Summary \u2192 Gotify",
|
||||
"description": null,
|
||||
"active": true,
|
||||
"isArchived": false,
|
||||
"nodes": [
|
||||
{
|
||||
"parameters": {
|
||||
"httpMethod": "POST",
|
||||
"path": "grafana-alert",
|
||||
"options": {}
|
||||
},
|
||||
"id": "a6e7dfb1-8e9b-4048-b028-4383c30a7e0d",
|
||||
"name": "Grafana Webhook",
|
||||
"type": "n8n-nodes-base.webhook",
|
||||
"typeVersion": 2,
|
||||
"position": [
|
||||
0,
|
||||
0
|
||||
],
|
||||
"webhookId": "0492788e-db4e-4d15-9572-9d6c3efc80c3"
|
||||
},
|
||||
{
|
||||
"parameters": {
|
||||
"jsCode": "// Parse Grafana alert payload into a clean summary for the LLM\nconst alerts = $input.first().json.body?.alerts || $input.first().json.alerts || [];\n\nconst parsed = alerts.map(a => ({\n status: a.status || 'unknown',\n alertName: a.labels?.alertname || 'Unnamed Alert',\n host: a.labels?.host || a.labels?.instance || 'unknown host',\n severity: a.labels?.severity || 'warning',\n summary: a.annotations?.summary || '',\n description: a.annotations?.description || '',\n value: a.valueString || '',\n startsAt: a.startsAt || '',\n endsAt: a.endsAt || ''\n}));\n\nconst alertText = parsed.map(a => \n `[${a.status.toUpperCase()}] ${a.alertName} on ${a.host}\\nSeverity: ${a.severity}\\nSummary: ${a.summary}\\nDescription: ${a.description}\\nValue: ${a.value}\\nStarted: ${a.startsAt}`\n).join('\\n---\\n');\n\nreturn [{\n json: {\n alertText,\n alertCount: parsed.length,\n hasResolved: parsed.some(a => a.status === 'resolved'),\n hasFiring: parsed.some(a => a.status === 'firing'),\n parsed\n }\n}];"
|
||||
},
|
||||
"id": "ea216058-69db-42cc-88d0-8f67e1b8951b",
|
||||
"name": "Parse Alert Payload",
|
||||
"type": "n8n-nodes-base.code",
|
||||
"typeVersion": 2,
|
||||
"position": [
|
||||
224,
|
||||
0
|
||||
]
|
||||
},
|
||||
{
|
||||
"parameters": {
|
||||
"method": "POST",
|
||||
"url": "http://10.5.1.6:4000/v1/chat/completions",
|
||||
"sendHeaders": true,
|
||||
"headerParameters": {
|
||||
"parameters": [
|
||||
{
|
||||
"name": "Content-Type",
|
||||
"value": "application/json"
|
||||
},
|
||||
{
|
||||
"name": "Authorization",
|
||||
"value": "Bearer sk-REPLACE_WITH_LITELLM_KEY"
|
||||
}
|
||||
]
|
||||
},
|
||||
"sendBody": true,
|
||||
"specifyBody": "json",
|
||||
"jsonBody": "={{\n {\n \"model\": \"medium\",\n \"messages\": [\n {\n \"role\": \"system\",\n \"content\": \"You are a concise homelab monitoring assistant. Summarize infrastructure alerts in 2-3 sentences. Include: what happened, which host, severity, and a suggested action. Keep it brief \u2014 this goes to a phone notification.\"\n },\n {\n \"role\": \"user\",\n \"content\": \"Summarize this Grafana alert:\\n\\n\" + $json.alertText\n }\n ],\n \"max_tokens\": 200,\n \"temperature\": 0.3\n }\n}}",
|
||||
"options": {}
|
||||
},
|
||||
"id": "8ef1aa9d-d5b1-41ea-abc6-703b6491ea33",
|
||||
"name": "LiteLLM Summarize",
|
||||
"type": "n8n-nodes-base.httpRequest",
|
||||
"typeVersion": 4.2,
|
||||
"position": [
|
||||
1792,
|
||||
0
|
||||
]
|
||||
},
|
||||
{
|
||||
"parameters": {
|
||||
"jsCode": "const response = $input.first().json;\nconst summary = response.choices?.[0]?.message?.content || 'Could not generate summary';\nconst alertData = $('Parse Alert Payload').first().json;\n\nconst title = alertData.hasFiring \n ? `\ud83d\udd25 Alert Firing (${alertData.alertCount})` \n : `\u2705 Alert Resolved (${alertData.alertCount})`;\n\nconst priority = alertData.hasFiring ? 8 : 2;\n\nreturn [{\n json: {\n title,\n message: summary,\n priority\n }\n}];"
|
||||
},
|
||||
"id": "2e969bca-8458-4477-a358-664fd1c50c4f",
|
||||
"name": "Format Notification",
|
||||
"type": "n8n-nodes-base.code",
|
||||
"typeVersion": 2,
|
||||
"position": [
|
||||
2016,
|
||||
0
|
||||
]
|
||||
},
|
||||
{
|
||||
"parameters": {
|
||||
"message": "={{ $json.message }}",
|
||||
"additionalFields": {
|
||||
"priority": "={{ $json.priority }}",
|
||||
"title": "={{ $json.title }}"
|
||||
},
|
||||
"options": {
|
||||
"contentType": "text/plain"
|
||||
}
|
||||
},
|
||||
"id": "65c2b53a-6e03-4460-9aa6-5d967d244de0",
|
||||
"name": "Push to Gotify",
|
||||
"type": "n8n-nodes-base.gotify",
|
||||
"typeVersion": 1,
|
||||
"position": [
|
||||
2224,
|
||||
0
|
||||
],
|
||||
"credentials": {
|
||||
"gotifyApi": {
|
||||
"id": "DXdYZDfVZecDTaNU",
|
||||
"name": "Gotify account"
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"parameters": {
|
||||
"jsCode": "const parsed = $('Parse Alert Payload').first().json;\nconst now = new Date();\nconst hour = now.getHours();\nconst fingerprint = (parsed.parsed || [])\n .map(a => `${a.alertName}:${a.host}`)\n .sort().join('|') || 'unknown';\nreturn [{ json: { ...parsed, fingerprint, isNightTime: hour >= 0 && hour < 7 } }];\n"
|
||||
},
|
||||
"id": "df26d9ca-c72c-4dee-9327-14a0f5fcf3a0",
|
||||
"name": "Build Fingerprint",
|
||||
"type": "n8n-nodes-base.code",
|
||||
"typeVersion": 2,
|
||||
"position": [
|
||||
448,
|
||||
0
|
||||
]
|
||||
},
|
||||
{
|
||||
"parameters": {
|
||||
"operation": "executeQuery",
|
||||
"query": "SELECT fingerprint, last_seen FROM grafana_alert_dedup WHERE fingerprint = '{{ $json.fingerprint }}' AND last_seen > NOW() - INTERVAL '30 minutes' LIMIT 1;",
|
||||
"options": {}
|
||||
},
|
||||
"id": "3fd78a53-7885-4739-a4b7-b441a27e993e",
|
||||
"name": "Lookup Dedup",
|
||||
"type": "n8n-nodes-base.postgres",
|
||||
"typeVersion": 2.5,
|
||||
"position": [
|
||||
672,
|
||||
0
|
||||
],
|
||||
"credentials": {
|
||||
"postgres": {
|
||||
"id": "n9svoXemqSZoNNUB",
|
||||
"name": "Postgres account"
|
||||
}
|
||||
},
|
||||
"continueOnFail": true
|
||||
},
|
||||
{
|
||||
"parameters": {
|
||||
"jsCode": "const alertData = $('Build Fingerprint').first().json;\nconst rows = $input.all().filter(r => r.json && r.json.fingerprint);\nconst alreadySeen = rows.length > 0;\nconst isLowPriority = !alertData.hasFiring;\nconst suppressNight = alertData.isNightTime && isLowPriority;\nconst suppress = alreadySeen || suppressNight;\nreturn [{ json: { ...alertData, suppress, suppressReason: alreadySeen ? 'dedup' : suppressNight ? 'night_suppress' : null } }];\n"
|
||||
},
|
||||
"id": "c34b4b0e-1a47-4624-b1c4-9ef6b5cdeb6c",
|
||||
"name": "Check Should Notify",
|
||||
"type": "n8n-nodes-base.code",
|
||||
"typeVersion": 2,
|
||||
"position": [
|
||||
896,
|
||||
0
|
||||
]
|
||||
},
|
||||
{
|
||||
"parameters": {
|
||||
"conditions": {
|
||||
"options": {
|
||||
"caseSensitive": true,
|
||||
"leftValue": "",
|
||||
"typeValidation": "strict"
|
||||
},
|
||||
"conditions": [
|
||||
{
|
||||
"id": "edee2fbd-0b2b-43bf-b542-9d9297d39d66",
|
||||
"leftValue": "={{ $json.suppress }}",
|
||||
"rightValue": false,
|
||||
"operator": {
|
||||
"type": "boolean",
|
||||
"operation": "equal"
|
||||
}
|
||||
}
|
||||
],
|
||||
"combinator": "and"
|
||||
}
|
||||
},
|
||||
"id": "29929dc8-17f6-4cc8-90c2-c501eaba8be5",
|
||||
"name": "Should Notify?",
|
||||
"type": "n8n-nodes-base.if",
|
||||
"typeVersion": 2.2,
|
||||
"position": [
|
||||
1120,
|
||||
0
|
||||
]
|
||||
},
|
||||
{
|
||||
"parameters": {
|
||||
"operation": "executeQuery",
|
||||
"query": "INSERT INTO grafana_alert_dedup (fingerprint, last_seen, alert_name, host) VALUES ('{{ $json.fingerprint }}', NOW(), '{{ ($json.parsed && $json.parsed[0]) ? $json.parsed[0].alertName : \"unknown\" }}', '{{ ($json.parsed && $json.parsed[0]) ? $json.parsed[0].host : \"unknown\" }}') ON CONFLICT (fingerprint) DO UPDATE SET last_seen = NOW();",
|
||||
"options": {}
|
||||
},
|
||||
"id": "25a14a6a-db17-49ca-a6d2-6ed2f97d2889",
|
||||
"name": "Upsert Dedup Record",
|
||||
"type": "n8n-nodes-base.postgres",
|
||||
"typeVersion": 2.5,
|
||||
"position": [
|
||||
1344,
|
||||
0
|
||||
],
|
||||
"credentials": {
|
||||
"postgres": {
|
||||
"id": "n9svoXemqSZoNNUB",
|
||||
"name": "Postgres account"
|
||||
}
|
||||
},
|
||||
"continueOnFail": true
|
||||
}
|
||||
],
|
||||
"connections": {
|
||||
"Grafana Webhook": {
|
||||
"main": [
|
||||
[
|
||||
{
|
||||
"node": "Parse Alert Payload",
|
||||
"type": "main",
|
||||
"index": 0
|
||||
}
|
||||
]
|
||||
]
|
||||
},
|
||||
"Parse Alert Payload": {
|
||||
"main": [
|
||||
[
|
||||
{
|
||||
"node": "Build Fingerprint",
|
||||
"type": "main",
|
||||
"index": 0
|
||||
}
|
||||
]
|
||||
]
|
||||
},
|
||||
"Build Fingerprint": {
|
||||
"main": [
|
||||
[
|
||||
{
|
||||
"node": "Lookup Dedup",
|
||||
"type": "main",
|
||||
"index": 0
|
||||
}
|
||||
]
|
||||
]
|
||||
},
|
||||
"Lookup Dedup": {
|
||||
"main": [
|
||||
[
|
||||
{
|
||||
"node": "Check Should Notify",
|
||||
"type": "main",
|
||||
"index": 0
|
||||
}
|
||||
]
|
||||
]
|
||||
},
|
||||
"Check Should Notify": {
|
||||
"main": [
|
||||
[
|
||||
{
|
||||
"node": "Should Notify?",
|
||||
"type": "main",
|
||||
"index": 0
|
||||
}
|
||||
]
|
||||
]
|
||||
},
|
||||
"Should Notify?": {
|
||||
"main": [
|
||||
[
|
||||
{
|
||||
"node": "Upsert Dedup Record",
|
||||
"type": "main",
|
||||
"index": 0
|
||||
}
|
||||
],
|
||||
[]
|
||||
]
|
||||
},
|
||||
"Upsert Dedup Record": {
|
||||
"main": [
|
||||
[
|
||||
{
|
||||
"node": "LiteLLM Summarize",
|
||||
"type": "main",
|
||||
"index": 0
|
||||
}
|
||||
]
|
||||
]
|
||||
},
|
||||
"LiteLLM Summarize": {
|
||||
"main": [
|
||||
[
|
||||
{
|
||||
"node": "Format Notification",
|
||||
"type": "main",
|
||||
"index": 0
|
||||
}
|
||||
]
|
||||
]
|
||||
},
|
||||
"Format Notification": {
|
||||
"main": [
|
||||
[
|
||||
{
|
||||
"node": "Push to Gotify",
|
||||
"type": "main",
|
||||
"index": 0
|
||||
}
|
||||
]
|
||||
]
|
||||
}
|
||||
},
|
||||
"settings": {
|
||||
"executionOrder": "v1",
|
||||
"binaryMode": "separate"
|
||||
},
|
||||
"staticData": null,
|
||||
"meta": {
|
||||
"templateCredsSetupCompleted": true
|
||||
},
|
||||
"pinData": {},
|
||||
"versionId": "ba3a4eab-60b8-4b1c-ac9f-07a0fdcec9c0",
|
||||
"activeVersionId": "ba3a4eab-60b8-4b1c-ac9f-07a0fdcec9c0",
|
||||
"versionCounter": 10,
|
||||
"triggerCount": 1,
|
||||
"tags": [
|
||||
{
|
||||
"updatedAt": "2026-05-06T22:38:51.678Z",
|
||||
"createdAt": "2026-05-06T22:38:51.678Z",
|
||||
"id": "S9FxzVfHLsHmyzyt",
|
||||
"name": "ai"
|
||||
},
|
||||
{
|
||||
"updatedAt": "2026-05-06T22:38:51.660Z",
|
||||
"createdAt": "2026-05-06T22:38:51.660Z",
|
||||
"id": "0ETpkL5jJ5wwgt8k",
|
||||
"name": "monitoring"
|
||||
}
|
||||
],
|
||||
"shared": [
|
||||
{
|
||||
"updatedAt": "2026-05-07T03:32:32.026Z",
|
||||
"createdAt": "2026-05-07T03:32:32.026Z",
|
||||
"role": "workflow:owner",
|
||||
"workflowId": "GXtkp4xKka90XrXH",
|
||||
"projectId": "dxCRnBdX5uJizCGa",
|
||||
"project": {
|
||||
"updatedAt": "2026-05-06T01:10:37.484Z",
|
||||
"createdAt": "2026-05-06T01:08:07.721Z",
|
||||
"id": "dxCRnBdX5uJizCGa",
|
||||
"name": "Wilfred Fizzlepoof <admin@paccoco.com>",
|
||||
"type": "personal",
|
||||
"icon": null,
|
||||
"description": null,
|
||||
"creatorId": "47b2227f-2fc2-4a08-bd35-ea157b92df0d"
|
||||
}
|
||||
}
|
||||
],
|
||||
"versionMetadata": {
|
||||
"name": "Version ba3a4eab",
|
||||
"description": ""
|
||||
}
|
||||
}
|
||||
394
n8n-workflows/02-rag-ingest-and-query.json
Normal file
394
n8n-workflows/02-rag-ingest-and-query.json
Normal file
@@ -0,0 +1,394 @@
|
||||
{
|
||||
"updatedAt": "2026-05-06T23:25:33.293Z",
|
||||
"createdAt": "2026-05-06T23:25:29.005Z",
|
||||
"id": "gYyOggC4J98keLbj",
|
||||
"name": "RAG Pipeline — Ingest & Query",
|
||||
"description": null,
|
||||
"active": true,
|
||||
"isArchived": false,
|
||||
"nodes": [
|
||||
{
|
||||
"parameters": {
|
||||
"httpMethod": "POST",
|
||||
"path": "rag/ingest",
|
||||
"responseMode": "lastNode",
|
||||
"responseData": "allEntries",
|
||||
"options": {}
|
||||
},
|
||||
"id": "31ce8413-d15c-490f-92d1-956ef214e9a3",
|
||||
"name": "Ingest Webhook",
|
||||
"type": "n8n-nodes-base.webhook",
|
||||
"typeVersion": 2,
|
||||
"position": [
|
||||
0,
|
||||
0
|
||||
],
|
||||
"webhookId": "411ad4e3-eb9f-49e2-97c4-2f30e01dd609"
|
||||
},
|
||||
{
|
||||
"parameters": {
|
||||
"jsCode": "// Split incoming text into chunks for embedding\nconst input = $input.first().json.body || $input.first().json;\nconst text = input.text || input.content || '';\nconst source = input.source || 'manual';\nconst metadata = input.metadata || {};\n\nconst CHUNK_SIZE = 500;\nconst OVERLAP = 50;\n\nconst chunks = [];\nconst sentences = text.split(/(?<=[.!?])\\s+/);\nlet current = '';\n\nfor (const sentence of sentences) {\n if ((current + ' ' + sentence).length > CHUNK_SIZE && current.length > 0) {\n chunks.push(current.trim());\n // Keep overlap\n const words = current.split(' ');\n current = words.slice(-OVERLAP).join(' ') + ' ' + sentence;\n } else {\n current = current ? current + ' ' + sentence : sentence;\n }\n}\nif (current.trim()) chunks.push(current.trim());\n\nreturn chunks.map((chunk, i) => ({\n json: {\n chunk,\n chunkIndex: i,\n totalChunks: chunks.length,\n source,\n metadata\n }\n}));"
|
||||
},
|
||||
"id": "5d5aa0b1-c84b-48f6-9b5e-618cb05e5e61",
|
||||
"name": "Chunk Text",
|
||||
"type": "n8n-nodes-base.code",
|
||||
"typeVersion": 2,
|
||||
"position": [
|
||||
224,
|
||||
0
|
||||
]
|
||||
},
|
||||
{
|
||||
"parameters": {
|
||||
"method": "POST",
|
||||
"url": "http://10.5.1.6:11434/v1/embeddings",
|
||||
"sendHeaders": true,
|
||||
"headerParameters": {
|
||||
"parameters": [
|
||||
{
|
||||
"name": "Content-Type",
|
||||
"value": "application/json"
|
||||
}
|
||||
]
|
||||
},
|
||||
"sendBody": true,
|
||||
"specifyBody": "json",
|
||||
"jsonBody": "={\n \"model\": \"nomic-embed-text\",\n \"input\": \"{{ $json.chunk }}\"\n}",
|
||||
"options": {
|
||||
"batching": {
|
||||
"batch": {
|
||||
"batchSize": 5,
|
||||
"batchInterval": 500
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"id": "89c692c6-b987-409c-b5c1-3533097d06e3",
|
||||
"name": "Get Embedding",
|
||||
"type": "n8n-nodes-base.httpRequest",
|
||||
"typeVersion": 4.2,
|
||||
"position": [
|
||||
448,
|
||||
0
|
||||
]
|
||||
},
|
||||
{
|
||||
"parameters": {
|
||||
"jsCode": "const embeddingResponse = $input.first().json;\nconst chunkData = $('Chunk Text').item;\nconst embedding = embeddingResponse.data?.[0]?.embedding || [];\n\nconst pointId = 'xxxxxxxx-xxxx-4xxx-yxxx-xxxxxxxxxxxx'.replace(/[xy]/g, c => { const r = Math.random() * 16 | 0; return (c === 'x' ? r : (r & 0x3 | 0x8)).toString(16); });\n\nreturn [{\n json: {\n id: pointId,\n vector: embedding,\n payload: {\n text: chunkData.json.chunk,\n source: chunkData.json.source,\n chunkIndex: chunkData.json.chunkIndex,\n totalChunks: chunkData.json.totalChunks,\n metadata: chunkData.json.metadata,\n ingested_at: new Date().toISOString()\n }\n }\n}];"
|
||||
},
|
||||
"id": "fec59066-958f-4b96-8224-6f4e48ad5abb",
|
||||
"name": "Prepare Qdrant Point",
|
||||
"type": "n8n-nodes-base.code",
|
||||
"typeVersion": 2,
|
||||
"position": [
|
||||
672,
|
||||
0
|
||||
]
|
||||
},
|
||||
{
|
||||
"parameters": {
|
||||
"method": "PUT",
|
||||
"url": "http://10.5.1.6:6333/collections/knowledge_base/points",
|
||||
"sendHeaders": true,
|
||||
"headerParameters": {
|
||||
"parameters": [
|
||||
{
|
||||
"name": "Content-Type",
|
||||
"value": "application/json"
|
||||
}
|
||||
]
|
||||
},
|
||||
"sendBody": true,
|
||||
"specifyBody": "json",
|
||||
"jsonBody": "={\n \"points\": [\n {\n \"id\": \"{{ $json.id }}\",\n \"vector\": {{ JSON.stringify($json.vector) }},\n \"payload\": {{ JSON.stringify($json.payload) }}\n }\n ]\n}",
|
||||
"options": {}
|
||||
},
|
||||
"id": "9404c8fe-46ae-4d8a-bf35-35ecba4939c0",
|
||||
"name": "Upsert to Qdrant",
|
||||
"type": "n8n-nodes-base.httpRequest",
|
||||
"typeVersion": 4.2,
|
||||
"position": [
|
||||
880,
|
||||
0
|
||||
]
|
||||
},
|
||||
{
|
||||
"parameters": {
|
||||
"httpMethod": "POST",
|
||||
"path": "rag/query",
|
||||
"responseMode": "lastNode",
|
||||
"responseData": "allEntries",
|
||||
"options": {}
|
||||
},
|
||||
"id": "1a5c921c-0351-4a5e-a3e4-0aab71313af1",
|
||||
"name": "Query Webhook",
|
||||
"type": "n8n-nodes-base.webhook",
|
||||
"typeVersion": 2,
|
||||
"position": [
|
||||
0,
|
||||
304
|
||||
],
|
||||
"webhookId": "aad4fe04-74b0-4136-80e9-380a7e4db1c2"
|
||||
},
|
||||
{
|
||||
"parameters": {
|
||||
"method": "POST",
|
||||
"url": "http://10.5.1.6:11434/v1/embeddings",
|
||||
"sendHeaders": true,
|
||||
"headerParameters": {
|
||||
"parameters": [
|
||||
{
|
||||
"name": "Content-Type",
|
||||
"value": "application/json"
|
||||
}
|
||||
]
|
||||
},
|
||||
"sendBody": true,
|
||||
"specifyBody": "json",
|
||||
"jsonBody": "={\n \"model\": \"nomic-embed-text\",\n \"input\": \"{{ $json.body.query || $json.query }}\"\n}",
|
||||
"options": {}
|
||||
},
|
||||
"id": "0a294ba5-a890-4dfa-8ae7-ba25cd0a4534",
|
||||
"name": "Embed Query",
|
||||
"type": "n8n-nodes-base.httpRequest",
|
||||
"typeVersion": 4.2,
|
||||
"position": [
|
||||
224,
|
||||
304
|
||||
]
|
||||
},
|
||||
{
|
||||
"parameters": {
|
||||
"method": "POST",
|
||||
"url": "http://10.5.1.6:6333/collections/knowledge_base/points/search",
|
||||
"sendHeaders": true,
|
||||
"headerParameters": {
|
||||
"parameters": [
|
||||
{
|
||||
"name": "Content-Type",
|
||||
"value": "application/json"
|
||||
}
|
||||
]
|
||||
},
|
||||
"sendBody": true,
|
||||
"specifyBody": "json",
|
||||
"jsonBody": "={\n \"vector\": {{ JSON.stringify($json.data[0].embedding) }},\n \"limit\": 5,\n \"with_payload\": true\n}",
|
||||
"options": {}
|
||||
},
|
||||
"id": "2cb68891-c57e-4006-b3ac-c290aec28220",
|
||||
"name": "Search Qdrant",
|
||||
"type": "n8n-nodes-base.httpRequest",
|
||||
"typeVersion": 4.2,
|
||||
"position": [
|
||||
448,
|
||||
304
|
||||
]
|
||||
},
|
||||
{
|
||||
"parameters": {
|
||||
"jsCode": "const searchResults = $input.first().json.result || [];\nconst query = $('Query Webhook').first().json.body?.query || $('Query Webhook').first().json.query || '';\n\nconst context = searchResults.map((r, i) => \n `[${i+1}] (score: ${r.score.toFixed(3)}) ${r.payload.text}`\n).join('\\n\\n');\n\nreturn [{\n json: {\n query,\n context,\n sources: searchResults.map(r => ({\n source: r.payload.source,\n score: r.score,\n text_preview: r.payload.text.substring(0, 100) + '...'\n }))\n }\n}];"
|
||||
},
|
||||
"id": "d32ee814-81d0-4ea6-8f28-59632d09b1b9",
|
||||
"name": "Build Context",
|
||||
"type": "n8n-nodes-base.code",
|
||||
"typeVersion": 2,
|
||||
"position": [
|
||||
672,
|
||||
304
|
||||
]
|
||||
},
|
||||
{
|
||||
"parameters": {
|
||||
"method": "POST",
|
||||
"url": "http://10.5.1.6:4000/v1/chat/completions",
|
||||
"sendHeaders": true,
|
||||
"headerParameters": {
|
||||
"parameters": [
|
||||
{
|
||||
"name": "Content-Type",
|
||||
"value": "application/json"
|
||||
},
|
||||
{
|
||||
"name": "Authorization",
|
||||
"value": "Bearer sk-REPLACE_WITH_LITELLM_KEY"
|
||||
}
|
||||
]
|
||||
},
|
||||
"sendBody": true,
|
||||
"specifyBody": "json",
|
||||
"jsonBody": "={\n \"model\": \"medium\",\n \"messages\": [\n {\n \"role\": \"system\",\n \"content\": \"You are a helpful knowledge base assistant. Answer the user's question using ONLY the provided context. If the context doesn't contain enough information, say so. Cite which source numbers you used.\"\n },\n {\n \"role\": \"user\",\n \"content\": \"Context:\\n{{ $json.context }}\\n\\nQuestion: {{ $json.query }}\"\n }\n ],\n \"max_tokens\": 500,\n \"temperature\": 0.2\n}",
|
||||
"options": {}
|
||||
},
|
||||
"id": "e1360454-8457-40ce-b179-670c182da59e",
|
||||
"name": "LiteLLM Answer",
|
||||
"type": "n8n-nodes-base.httpRequest",
|
||||
"typeVersion": 4.2,
|
||||
"position": [
|
||||
880,
|
||||
304
|
||||
]
|
||||
},
|
||||
{
|
||||
"parameters": {
|
||||
"jsCode": "const response = $input.first().json;\nconst contextData = $('Build Context').first().json;\n\nreturn [{\n json: {\n answer: response.choices?.[0]?.message?.content || 'No answer generated',\n sources: contextData.sources,\n query: contextData.query\n }\n}];"
|
||||
},
|
||||
"id": "fe16d3a1-0d11-442f-b941-56fb74804601",
|
||||
"name": "Format Response",
|
||||
"type": "n8n-nodes-base.code",
|
||||
"typeVersion": 2,
|
||||
"position": [
|
||||
1104,
|
||||
304
|
||||
]
|
||||
}
|
||||
],
|
||||
"connections": {
|
||||
"Ingest Webhook": {
|
||||
"main": [
|
||||
[
|
||||
{
|
||||
"node": "Chunk Text",
|
||||
"type": "main",
|
||||
"index": 0
|
||||
}
|
||||
]
|
||||
]
|
||||
},
|
||||
"Chunk Text": {
|
||||
"main": [
|
||||
[
|
||||
{
|
||||
"node": "Get Embedding",
|
||||
"type": "main",
|
||||
"index": 0
|
||||
}
|
||||
]
|
||||
]
|
||||
},
|
||||
"Get Embedding": {
|
||||
"main": [
|
||||
[
|
||||
{
|
||||
"node": "Prepare Qdrant Point",
|
||||
"type": "main",
|
||||
"index": 0
|
||||
}
|
||||
]
|
||||
]
|
||||
},
|
||||
"Prepare Qdrant Point": {
|
||||
"main": [
|
||||
[
|
||||
{
|
||||
"node": "Upsert to Qdrant",
|
||||
"type": "main",
|
||||
"index": 0
|
||||
}
|
||||
]
|
||||
]
|
||||
},
|
||||
"Query Webhook": {
|
||||
"main": [
|
||||
[
|
||||
{
|
||||
"node": "Embed Query",
|
||||
"type": "main",
|
||||
"index": 0
|
||||
}
|
||||
]
|
||||
]
|
||||
},
|
||||
"Embed Query": {
|
||||
"main": [
|
||||
[
|
||||
{
|
||||
"node": "Search Qdrant",
|
||||
"type": "main",
|
||||
"index": 0
|
||||
}
|
||||
]
|
||||
]
|
||||
},
|
||||
"Search Qdrant": {
|
||||
"main": [
|
||||
[
|
||||
{
|
||||
"node": "Build Context",
|
||||
"type": "main",
|
||||
"index": 0
|
||||
}
|
||||
]
|
||||
]
|
||||
},
|
||||
"Build Context": {
|
||||
"main": [
|
||||
[
|
||||
{
|
||||
"node": "LiteLLM Answer",
|
||||
"type": "main",
|
||||
"index": 0
|
||||
}
|
||||
]
|
||||
]
|
||||
},
|
||||
"LiteLLM Answer": {
|
||||
"main": [
|
||||
[
|
||||
{
|
||||
"node": "Format Response",
|
||||
"type": "main",
|
||||
"index": 0
|
||||
}
|
||||
]
|
||||
]
|
||||
}
|
||||
},
|
||||
"settings": {
|
||||
"executionOrder": "v1",
|
||||
"binaryMode": "separate"
|
||||
},
|
||||
"staticData": null,
|
||||
"meta": null,
|
||||
"pinData": {},
|
||||
"versionId": "832248e5-106e-4fc4-8ebd-a5d0034e2b14",
|
||||
"activeVersionId": "832248e5-106e-4fc4-8ebd-a5d0034e2b14",
|
||||
"versionCounter": 14,
|
||||
"triggerCount": 2,
|
||||
"tags": [
|
||||
{
|
||||
"updatedAt": "2026-05-06T22:38:51.678Z",
|
||||
"createdAt": "2026-05-06T22:38:51.678Z",
|
||||
"id": "S9FxzVfHLsHmyzyt",
|
||||
"name": "ai"
|
||||
},
|
||||
{
|
||||
"updatedAt": "2026-05-06T22:39:33.284Z",
|
||||
"createdAt": "2026-05-06T22:39:33.284Z",
|
||||
"id": "7uuEyQrIlcuruZBk",
|
||||
"name": "rag"
|
||||
}
|
||||
],
|
||||
"shared": [
|
||||
{
|
||||
"updatedAt": "2026-05-06T23:25:29.005Z",
|
||||
"createdAt": "2026-05-06T23:25:29.005Z",
|
||||
"role": "workflow:owner",
|
||||
"workflowId": "gYyOggC4J98keLbj",
|
||||
"projectId": "dxCRnBdX5uJizCGa",
|
||||
"project": {
|
||||
"updatedAt": "2026-05-06T01:10:37.484Z",
|
||||
"createdAt": "2026-05-06T01:08:07.721Z",
|
||||
"id": "dxCRnBdX5uJizCGa",
|
||||
"name": "Wilfred Fizzlepoof <admin@paccoco.com>",
|
||||
"type": "personal",
|
||||
"icon": null,
|
||||
"description": null,
|
||||
"creatorId": "47b2227f-2fc2-4a08-bd35-ea157b92df0d"
|
||||
}
|
||||
}
|
||||
],
|
||||
"versionMetadata": {
|
||||
"name": "Version 832248e5",
|
||||
"description": ""
|
||||
}
|
||||
}
|
||||
348
n8n-workflows/03-paperless-ai-processing.json
Normal file
348
n8n-workflows/03-paperless-ai-processing.json
Normal file
@@ -0,0 +1,348 @@
|
||||
{
|
||||
"updatedAt": "2026-05-07T20:24:47.896Z",
|
||||
"createdAt": "2026-05-07T20:24:40.153Z",
|
||||
"id": "dQ1d5deV4mF0s5eP",
|
||||
"name": "Paperless-NGX → AI Processing → Native Gotify + Apply Tags",
|
||||
"description": null,
|
||||
"active": true,
|
||||
"isArchived": false,
|
||||
"nodes": [
|
||||
{
|
||||
"parameters": {
|
||||
"httpMethod": "POST",
|
||||
"path": "paperless/new-document",
|
||||
"options": {}
|
||||
},
|
||||
"id": "3d8bc021-e27f-4f7b-b716-a202d66551be",
|
||||
"name": "Paperless Webhook",
|
||||
"type": "n8n-nodes-base.webhook",
|
||||
"typeVersion": 2,
|
||||
"position": [
|
||||
0,
|
||||
0
|
||||
],
|
||||
"webhookId": "fd1eaa3a-fbf1-4a17-94fd-bb483df602b7"
|
||||
},
|
||||
{
|
||||
"parameters": {
|
||||
"jsCode": "// Extract document ID from whatever Paperless sends\nconst data = $input.first().json;\nconst body = data.body || data;\n\n// Try multiple possible field names and formats\nlet docId = body.document_id || body.id || body.doc_id || body.pk;\n\n// If it's a string like '{document_id}' (template not resolved), fall back to latest doc\nif (!docId || typeof docId === 'string' && docId.includes('{')) {\n docId = null;\n}\n\nreturn [{ json: { docId } }];"
|
||||
},
|
||||
"id": "48277e9e-a835-4e12-8608-63e905abe74d",
|
||||
"name": "Extract Document ID",
|
||||
"type": "n8n-nodes-base.code",
|
||||
"typeVersion": 2,
|
||||
"position": [
|
||||
224,
|
||||
0
|
||||
]
|
||||
},
|
||||
{
|
||||
"parameters": {
|
||||
"url": "=https://paperless.paccoco.com/api/documents/{{ $json.docId ? $json.docId + '/' : '?ordering=-added&page_size=1' }}",
|
||||
"sendHeaders": true,
|
||||
"headerParameters": {
|
||||
"parameters": [
|
||||
{
|
||||
"name": "Authorization",
|
||||
"value": "Token 4fb34ba35b4ab76a0715e8c56faf00a31de4fb5d"
|
||||
}
|
||||
]
|
||||
},
|
||||
"options": {}
|
||||
},
|
||||
"id": "17ed7e6f-ccaa-43d3-9031-9f3d2bb12dec",
|
||||
"name": "Fetch Document",
|
||||
"type": "n8n-nodes-base.httpRequest",
|
||||
"typeVersion": 4.2,
|
||||
"position": [
|
||||
448,
|
||||
0
|
||||
]
|
||||
},
|
||||
{
|
||||
"parameters": {
|
||||
"jsCode": "const response = $input.first().json;\n\n// Handle both single doc response and list response\nconst doc = response.results ? response.results[0] : response;\n\nif (!doc || !doc.id) {\n throw new Error('No document found in response');\n}\n\n// Content is already in the metadata from Paperless OCR\nconst content = (doc.content || '').substring(0, 4000);\n\nreturn [{\n json: {\n documentId: doc.id,\n title: doc.title || 'Untitled',\n originalFilename: doc.original_file_name || '',\n correspondent: doc.correspondent_name || null,\n content: content,\n created: doc.created || new Date().toISOString()\n }\n}];"
|
||||
},
|
||||
"id": "3a026b3b-141b-48b4-a9ae-03998914f45c",
|
||||
"name": "Prepare Content",
|
||||
"type": "n8n-nodes-base.code",
|
||||
"typeVersion": 2,
|
||||
"position": [
|
||||
672,
|
||||
0
|
||||
]
|
||||
},
|
||||
{
|
||||
"parameters": {
|
||||
"method": "POST",
|
||||
"url": "http://10.5.1.6:4000/v1/chat/completions",
|
||||
"sendHeaders": true,
|
||||
"headerParameters": {
|
||||
"parameters": [
|
||||
{
|
||||
"name": "Content-Type",
|
||||
"value": "application/json"
|
||||
},
|
||||
{
|
||||
"name": "Authorization",
|
||||
"value": "Bearer sk-REPLACE_WITH_LITELLM_KEY"
|
||||
}
|
||||
]
|
||||
},
|
||||
"sendBody": true,
|
||||
"specifyBody": "json",
|
||||
"jsonBody": "={\n \"model\": \"medium\",\n \"messages\": [\n {\n \"role\": \"system\",\n \"content\": \"You are a document classification assistant. Analyze the document and return ONLY valid JSON. Do not include markdown. Do not include code fences.\\n\\nReturn this exact JSON structure:\\n{\\n \\\"summary\\\": \\\"short 1-2 sentence summary with no sensitive identifiers\\\",\\n \\\"doc_type\\\": \\\"invoice | receipt | letter | contract | manual | report | medical | tax | military | other\\\",\\n \\\"suggested_tags\\\": [\\\"tag1\\\", \\\"tag2\\\", \\\"tag3\\\"]\\n}\\n\\nIf the document is sensitive, summarize only generally and say sensitive personal identifiers were present and redacted. If the document is not sensitive, provide a normal 1-2 sentence summary. Never mention or summarize sensitive field categories such as SSNs, dates of birth, addresses, phone numbers, financial amounts, ID numbers, service numbers, or exact dates.\"\n },\n {\n \"role\": \"user\",\n \"content\": \"{{ JSON.stringify('Document title: ' + ($json.title || 'Untitled') + '\\nFilename: ' + ($json.originalFilename || '') + '\\nCorrespondent: ' + ($json.correspondent || 'Unknown') + '\\n\\nContent:\\n' + ($json.content || '')).slice(1, -1) }}\"\n }\n ],\n \"max_tokens\": 350,\n \"temperature\": 0.1,\n \"response_format\": {\n \"type\": \"json_object\"\n }\n}",
|
||||
"options": {}
|
||||
},
|
||||
"id": "51374897-d90e-49a2-889e-0476817dd9d8",
|
||||
"name": "AI Classify & Extract",
|
||||
"type": "n8n-nodes-base.httpRequest",
|
||||
"typeVersion": 4.2,
|
||||
"position": [
|
||||
880,
|
||||
0
|
||||
]
|
||||
},
|
||||
{
|
||||
"parameters": {
|
||||
"jsCode": "const response = $input.first().json;\nconst docData = $('Prepare Content').first().json;\n\nconst raw = response.choices?.[0]?.message?.content || '{}';\nlet analysis;\n\ntry {\n analysis = JSON.parse(raw);\n} catch (e) {\n analysis = {\n summary: 'Document processed, but AI response could not be parsed cleanly. Sensitive details were not included.',\n doc_type: 'other',\n suggested_tags: []\n };\n}\n\nfunction cleanText(value) {\n return String(value || '')\n .replace(/\\b\\d{3}[- ]?\\d{2}[- ]?\\d{4}\\b/g, '[REDACTED_SSN]')\n .replace(/\\b\\d{8}\\b/g, '[REDACTED_DATE]')\n .replace(/\\b\\d{1,5}\\s+[A-Za-z0-9 .'-]+\\s+(Road|RD|Street|St|Avenue|Ave|Drive|Dr|Lane|Ln|Court|Ct|Branch|Blvd|Boulevard)\\b[^,\\n]*/gi, '[REDACTED_ADDRESS]')\n .replace(/\\b\\d{5}(?:-\\d{4})?\\b/g, '[REDACTED_ZIP]');\n}\n\nconst tags = Array.isArray(analysis.suggested_tags)\n ? analysis.suggested_tags\n .map(t => cleanText(t).trim().toLowerCase())\n .filter(Boolean)\n .map(t => t.replace(/[^a-z0-9 _-]/gi, '').trim())\n .filter(Boolean)\n .slice(0, 5)\n : [];\n\nreturn [\n {\n json: {\n documentId: docData.documentId,\n title: cleanText(docData.title || 'Untitled'),\n summary: cleanText(analysis.summary || 'Document processed.'),\n doc_type: cleanText(analysis.doc_type || 'other').toLowerCase(),\n suggested_tags: tags\n }\n }\n];"
|
||||
},
|
||||
"id": "1f37f096-132b-43b4-bcb4-f5ed094bc9e5",
|
||||
"name": "Parse Analysis",
|
||||
"type": "n8n-nodes-base.code",
|
||||
"typeVersion": 2,
|
||||
"position": [
|
||||
1104,
|
||||
0
|
||||
]
|
||||
},
|
||||
{
|
||||
"parameters": {
|
||||
"method": "POST",
|
||||
"url": "=https://paperless.paccoco.com/api/documents/{{ $json.documentId }}/notes/",
|
||||
"sendHeaders": true,
|
||||
"headerParameters": {
|
||||
"parameters": [
|
||||
{
|
||||
"name": "Authorization",
|
||||
"value": "Token 4fb34ba35b4ab76a0715e8c56faf00a31de4fb5d"
|
||||
},
|
||||
{
|
||||
"name": "Content-Type",
|
||||
"value": "application/json"
|
||||
}
|
||||
]
|
||||
},
|
||||
"sendBody": true,
|
||||
"specifyBody": "json",
|
||||
"jsonBody": "={\n \"note\": \"{{ JSON.stringify('**AI Analysis**\\nType: ' + ($json.doc_type || 'other') + '\\nSummary: ' + ($json.summary || '') + '\\nSuggested tags: ' + (($json.suggested_tags || []).join(', '))).slice(1, -1) }}\"\n}",
|
||||
"options": {}
|
||||
},
|
||||
"id": "5597c8c1-436e-4373-bb54-0180be4e7bca",
|
||||
"name": "Add Note to Document",
|
||||
"type": "n8n-nodes-base.httpRequest",
|
||||
"typeVersion": 4.2,
|
||||
"position": [
|
||||
1328,
|
||||
0
|
||||
]
|
||||
},
|
||||
{
|
||||
"parameters": {
|
||||
"jsCode": "const data = $input.first().json;\n\nconst tags = Array.isArray(data.suggested_tags)\n ? data.suggested_tags.join(', ')\n : 'None';\n\nconst message = [\n data.title || 'Untitled',\n '',\n `Type: ${data.doc_type || 'other'}`,\n '',\n `Summary: ${data.summary || 'No summary returned'}`,\n '',\n `Suggested tags: ${tags || 'None'}`\n].join('\\n');\n\nreturn [{\n json: {\n title: '📄 New Document Processed',\n message,\n priority: 4\n }\n}];"
|
||||
},
|
||||
"id": "c3319471-e3f1-4cd3-aae1-c794f081b1e3",
|
||||
"name": "Format Gotify Document Message",
|
||||
"type": "n8n-nodes-base.code",
|
||||
"typeVersion": 2,
|
||||
"position": [
|
||||
1328,
|
||||
208
|
||||
]
|
||||
},
|
||||
{
|
||||
"parameters": {
|
||||
"message": "={{ $json.message }}",
|
||||
"additionalFields": {
|
||||
"priority": "={{ $json.priority }}",
|
||||
"title": "={{ $json.title }}"
|
||||
},
|
||||
"options": {
|
||||
"contentType": "text/plain"
|
||||
}
|
||||
},
|
||||
"id": "193f19dc-2ad6-41af-bd29-a0f5dc0ef1a0",
|
||||
"name": "Send Gotify Message",
|
||||
"type": "n8n-nodes-base.gotify",
|
||||
"typeVersion": 1,
|
||||
"position": [
|
||||
1552,
|
||||
208
|
||||
],
|
||||
"credentials": {
|
||||
"gotifyApi": {
|
||||
"id": "ajvRjvj0QldLQYmo",
|
||||
"name": "Paperless"
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"parameters": {
|
||||
"jsCode": "const data = $input.first().json;\n\nconst PAPERLESS_BASE_URL = 'https://paperless.paccoco.com';\nconst PAPERLESS_TOKEN = '4fb34ba35b4ab76a0715e8c56faf00a31de4fb5d';\n\nconst headers = {\n Authorization: `Token ${PAPERLESS_TOKEN}`,\n 'Content-Type': 'application/json',\n};\n\nfunction normalizeTagName(tag) {\n return String(tag || '')\n .trim()\n .toLowerCase()\n .replace(/[^a-z0-9 _-]/gi, '')\n .replace(/\\s+/g, ' ')\n .slice(0, 64);\n}\n\nconst suggestedTags = Array.isArray(data.suggested_tags)\n ? [...new Set(data.suggested_tags.map(normalizeTagName).filter(Boolean))].slice(0, 5)\n : [];\n\nif (!data.documentId) {\n throw new Error('No documentId found; cannot apply tags.');\n}\n\nif (suggestedTags.length === 0) {\n return [{\n json: {\n ...data,\n tag_apply_status: 'skipped_no_suggested_tags',\n applied_tag_names: [],\n applied_tag_ids: []\n }\n }];\n}\n\nasync function request(method, url, body) {\n const options = {\n method,\n url,\n headers,\n json: true,\n };\n\n if (body !== undefined) {\n options.body = body;\n }\n\n return await this.helpers.httpRequest(options);\n}\n\n// Fetch all existing tags. Handles pagination.\nconst existingTags = [];\nlet tagsUrl = `${PAPERLESS_BASE_URL}/api/tags/?page_size=100`;\n\nwhile (tagsUrl) {\n const page = await request.call(this, 'GET', tagsUrl);\n const results = Array.isArray(page.results) ? page.results : (Array.isArray(page) ? page : []);\n existingTags.push(...results);\n\n if (page.next) {\n tagsUrl = page.next.startsWith('http') ? page.next : `${PAPERLESS_BASE_URL}${page.next}`;\n } else {\n tagsUrl = null;\n }\n}\n\nconst tagsByName = new Map(\n existingTags.map(tag => [normalizeTagName(tag.name), tag])\n);\n\nconst appliedTags = [];\n\nfor (const tagName of suggestedTags) {\n let tag = tagsByName.get(tagName);\n\n if (!tag) {\n tag = await request.call(this, 'POST', `${PAPERLESS_BASE_URL}/api/tags/`, {\n name: tagName,\n });\n tagsByName.set(tagName, tag);\n }\n\n if (tag?.id) {\n appliedTags.push(tag);\n }\n}\n\n// Fetch current document so we merge tags instead of overwriting existing ones.\nconst doc = await request.call(this, 'GET', `${PAPERLESS_BASE_URL}/api/documents/${data.documentId}/`);\nconst currentTagIds = Array.isArray(doc.tags) ? doc.tags : [];\nconst appliedTagIds = appliedTags.map(tag => tag.id);\nconst mergedTagIds = [...new Set([...currentTagIds, ...appliedTagIds])];\n\nawait request.call(this, 'PATCH', `${PAPERLESS_BASE_URL}/api/documents/${data.documentId}/`, {\n tags: mergedTagIds,\n});\n\nreturn [{\n json: {\n ...data,\n tag_apply_status: 'updated',\n applied_tag_names: appliedTags.map(tag => tag.name),\n applied_tag_ids: appliedTagIds,\n final_tag_ids: mergedTagIds,\n }\n}];"
|
||||
},
|
||||
"id": "b0d5423b-dc83-4adf-98c6-d95afa3e5c1b",
|
||||
"name": "Apply Suggested Tags to Paperless",
|
||||
"type": "n8n-nodes-base.code",
|
||||
"typeVersion": 2,
|
||||
"position": [
|
||||
1328,
|
||||
400
|
||||
]
|
||||
}
|
||||
],
|
||||
"connections": {
|
||||
"Paperless Webhook": {
|
||||
"main": [
|
||||
[
|
||||
{
|
||||
"node": "Extract Document ID",
|
||||
"type": "main",
|
||||
"index": 0
|
||||
}
|
||||
]
|
||||
]
|
||||
},
|
||||
"Extract Document ID": {
|
||||
"main": [
|
||||
[
|
||||
{
|
||||
"node": "Fetch Document",
|
||||
"type": "main",
|
||||
"index": 0
|
||||
}
|
||||
]
|
||||
]
|
||||
},
|
||||
"Fetch Document": {
|
||||
"main": [
|
||||
[
|
||||
{
|
||||
"node": "Prepare Content",
|
||||
"type": "main",
|
||||
"index": 0
|
||||
}
|
||||
]
|
||||
]
|
||||
},
|
||||
"Prepare Content": {
|
||||
"main": [
|
||||
[
|
||||
{
|
||||
"node": "AI Classify & Extract",
|
||||
"type": "main",
|
||||
"index": 0
|
||||
}
|
||||
]
|
||||
]
|
||||
},
|
||||
"AI Classify & Extract": {
|
||||
"main": [
|
||||
[
|
||||
{
|
||||
"node": "Parse Analysis",
|
||||
"type": "main",
|
||||
"index": 0
|
||||
}
|
||||
]
|
||||
]
|
||||
},
|
||||
"Parse Analysis": {
|
||||
"main": [
|
||||
[
|
||||
{
|
||||
"node": "Add Note to Document",
|
||||
"type": "main",
|
||||
"index": 0
|
||||
},
|
||||
{
|
||||
"node": "Format Gotify Document Message",
|
||||
"type": "main",
|
||||
"index": 0
|
||||
},
|
||||
{
|
||||
"node": "Apply Suggested Tags to Paperless",
|
||||
"type": "main",
|
||||
"index": 0
|
||||
}
|
||||
]
|
||||
]
|
||||
},
|
||||
"Format Gotify Document Message": {
|
||||
"main": [
|
||||
[
|
||||
{
|
||||
"node": "Send Gotify Message",
|
||||
"type": "main",
|
||||
"index": 0
|
||||
}
|
||||
]
|
||||
]
|
||||
}
|
||||
},
|
||||
"settings": {
|
||||
"executionOrder": "v1",
|
||||
"binaryMode": "separate"
|
||||
},
|
||||
"staticData": null,
|
||||
"meta": {
|
||||
"templateCredsSetupCompleted": true
|
||||
},
|
||||
"pinData": {},
|
||||
"versionId": "199d2609-e3e7-46aa-b144-df038eb10a0d",
|
||||
"activeVersionId": "199d2609-e3e7-46aa-b144-df038eb10a0d",
|
||||
"versionCounter": 11,
|
||||
"triggerCount": 1,
|
||||
"tags": [
|
||||
{
|
||||
"updatedAt": "2026-05-06T22:39:46.433Z",
|
||||
"createdAt": "2026-05-06T22:39:46.433Z",
|
||||
"id": "dZUptsMy60GbLHft",
|
||||
"name": "documents"
|
||||
},
|
||||
{
|
||||
"updatedAt": "2026-05-06T22:38:51.678Z",
|
||||
"createdAt": "2026-05-06T22:38:51.678Z",
|
||||
"id": "S9FxzVfHLsHmyzyt",
|
||||
"name": "ai"
|
||||
},
|
||||
{
|
||||
"updatedAt": "2026-05-06T22:39:46.476Z",
|
||||
"createdAt": "2026-05-06T22:39:46.476Z",
|
||||
"id": "NxHjuO4MMd5kdPR7",
|
||||
"name": "paperless"
|
||||
}
|
||||
],
|
||||
"shared": [
|
||||
{
|
||||
"updatedAt": "2026-05-07T20:24:40.153Z",
|
||||
"createdAt": "2026-05-07T20:24:40.153Z",
|
||||
"role": "workflow:owner",
|
||||
"workflowId": "dQ1d5deV4mF0s5eP",
|
||||
"projectId": "dxCRnBdX5uJizCGa",
|
||||
"project": {
|
||||
"updatedAt": "2026-05-06T01:10:37.484Z",
|
||||
"createdAt": "2026-05-06T01:08:07.721Z",
|
||||
"id": "dxCRnBdX5uJizCGa",
|
||||
"name": "Wilfred Fizzlepoof <admin@paccoco.com>",
|
||||
"type": "personal",
|
||||
"icon": null,
|
||||
"description": null,
|
||||
"creatorId": "47b2227f-2fc2-4a08-bd35-ea157b92df0d"
|
||||
}
|
||||
}
|
||||
],
|
||||
"versionMetadata": {
|
||||
"name": "Version 199d2609",
|
||||
"description": ""
|
||||
}
|
||||
}
|
||||
589
n8n-workflows/04-whisper-transcription.json
Normal file
589
n8n-workflows/04-whisper-transcription.json
Normal file
@@ -0,0 +1,589 @@
|
||||
{
|
||||
"updatedAt": "2026-05-06T23:30:36.237Z",
|
||||
"createdAt": "2026-05-06T23:30:36.237Z",
|
||||
"id": "QEVnqyN4h5kcpVuJ",
|
||||
"name": "Whisper Audio Transcription",
|
||||
"description": null,
|
||||
"active": true,
|
||||
"isArchived": false,
|
||||
"nodes": [
|
||||
{
|
||||
"parameters": {
|
||||
"httpMethod": "POST",
|
||||
"path": "transcribe",
|
||||
"responseMode": "lastNode",
|
||||
"responseData": "allEntries",
|
||||
"options": {
|
||||
"rawBody": true
|
||||
}
|
||||
},
|
||||
"id": "91945dd9-42f8-4343-b131-9e46272f04bf",
|
||||
"name": "Transcription Webhook",
|
||||
"type": "n8n-nodes-base.webhook",
|
||||
"typeVersion": 2,
|
||||
"position": [
|
||||
0,
|
||||
0
|
||||
],
|
||||
"webhookId": "f50eec94-1e56-4189-88ee-51b701c1d889"
|
||||
},
|
||||
{
|
||||
"parameters": {
|
||||
"method": "POST",
|
||||
"url": "http://10.5.1.6:8786/v1/audio/transcriptions",
|
||||
"sendHeaders": true,
|
||||
"headerParameters": {
|
||||
"parameters": [
|
||||
{
|
||||
"name": "Accept",
|
||||
"value": "application/json"
|
||||
}
|
||||
]
|
||||
},
|
||||
"sendBody": true,
|
||||
"contentType": "multipart-form-data",
|
||||
"bodyParameters": {
|
||||
"parameters": [
|
||||
{
|
||||
"parameterType": "formBinaryData",
|
||||
"name": "file",
|
||||
"inputDataFieldName": "data"
|
||||
},
|
||||
{
|
||||
"name": "model",
|
||||
"value": "Systran/faster-whisper-large-v3"
|
||||
},
|
||||
{
|
||||
"name": "language",
|
||||
"value": "en"
|
||||
},
|
||||
{
|
||||
"name": "response_format",
|
||||
"value": "verbose_json"
|
||||
}
|
||||
]
|
||||
},
|
||||
"options": {}
|
||||
},
|
||||
"id": "fc726196-f62d-4ff4-9be6-a84b9dbfff9e",
|
||||
"name": "Whisper Transcribe",
|
||||
"type": "n8n-nodes-base.httpRequest",
|
||||
"typeVersion": 4.2,
|
||||
"position": [
|
||||
224,
|
||||
0
|
||||
]
|
||||
},
|
||||
{
|
||||
"parameters": {
|
||||
"jsCode": "const result = $input.first().json;\n\nconst segments = result.segments || [];\nconst fullText = result.text || segments.map(s => s.text).join(' ');\nconst duration = result.duration || segments[segments.length - 1]?.end || 0;\n\n// Format timestamps\nconst timestamped = segments.map(s => {\n const start = new Date(s.start * 1000).toISOString().substr(11, 8);\n return `[${start}] ${s.text.trim()}`;\n}).join('\\n');\n\nreturn [{\n json: {\n text: fullText.trim(),\n timestamped_text: timestamped,\n duration_seconds: Math.round(duration),\n segment_count: segments.length,\n language: result.language || 'en'\n }\n}];"
|
||||
},
|
||||
"id": "1b6f017f-849d-456f-9e4b-a0bced1d8e9f",
|
||||
"name": "Format Transcript",
|
||||
"type": "n8n-nodes-base.code",
|
||||
"typeVersion": 2,
|
||||
"position": [
|
||||
448,
|
||||
0
|
||||
]
|
||||
},
|
||||
{
|
||||
"parameters": {
|
||||
"httpMethod": "POST",
|
||||
"path": "transcribe/summarize",
|
||||
"responseMode": "lastNode",
|
||||
"responseData": "allEntries",
|
||||
"options": {
|
||||
"rawBody": true
|
||||
}
|
||||
},
|
||||
"id": "c2f8a2dd-cbbc-4b62-ae24-0f4a95bdf862",
|
||||
"name": "Transcribe + Summarize Webhook",
|
||||
"type": "n8n-nodes-base.webhook",
|
||||
"typeVersion": 2,
|
||||
"position": [
|
||||
0,
|
||||
304
|
||||
],
|
||||
"webhookId": "69b4b2e8-9486-4aa3-b9db-fb8e6bdae98e"
|
||||
},
|
||||
{
|
||||
"parameters": {
|
||||
"method": "POST",
|
||||
"url": "http://10.5.1.6:8786/v1/audio/transcriptions",
|
||||
"sendHeaders": true,
|
||||
"headerParameters": {
|
||||
"parameters": [
|
||||
{
|
||||
"name": "Accept",
|
||||
"value": "application/json"
|
||||
}
|
||||
]
|
||||
},
|
||||
"sendBody": true,
|
||||
"contentType": "multipart-form-data",
|
||||
"bodyParameters": {
|
||||
"parameters": [
|
||||
{
|
||||
"parameterType": "formBinaryData",
|
||||
"name": "file",
|
||||
"inputDataFieldName": "data"
|
||||
},
|
||||
{
|
||||
"name": "model",
|
||||
"value": "Systran/faster-whisper-large-v3"
|
||||
},
|
||||
{
|
||||
"name": "language",
|
||||
"value": "en"
|
||||
}
|
||||
]
|
||||
},
|
||||
"options": {}
|
||||
},
|
||||
"id": "ee25bf40-99ef-472b-928d-6ef63516094c",
|
||||
"name": "Whisper Transcribe 2",
|
||||
"type": "n8n-nodes-base.httpRequest",
|
||||
"typeVersion": 4.2,
|
||||
"position": [
|
||||
224,
|
||||
304
|
||||
]
|
||||
},
|
||||
{
|
||||
"parameters": {
|
||||
"method": "POST",
|
||||
"url": "http://10.5.1.6:4000/v1/chat/completions",
|
||||
"sendHeaders": true,
|
||||
"headerParameters": {
|
||||
"parameters": [
|
||||
{
|
||||
"name": "Content-Type",
|
||||
"value": "application/json"
|
||||
},
|
||||
{
|
||||
"name": "Authorization",
|
||||
"value": "Bearer sk-REPLACE_WITH_LITELLM_KEY"
|
||||
}
|
||||
]
|
||||
},
|
||||
"sendBody": true,
|
||||
"specifyBody": "json",
|
||||
"jsonBody": "={\n \"model\": \"medium\",\n \"messages\": [\n {\n \"role\": \"system\",\n \"content\": \"You are a meeting/audio note summarizer. Given a transcription, provide:\\n1. A brief summary (2-3 sentences)\\n2. Key points as a bullet list\\n3. Any action items mentioned\\n\\nRespond in JSON with keys: summary, key_points (array), action_items (array)\"\n },\n {\n \"role\": \"user\",\n \"content\": \"Summarize this transcription:\\n\\n{{ $json.text }}\"\n }\n ],\n \"max_tokens\": 500,\n \"temperature\": 0.2,\n \"response_format\": { \"type\": \"json_object\" }\n}",
|
||||
"options": {}
|
||||
},
|
||||
"id": "eda24425-dc26-4b4b-858e-9ae0d5e1a5e4",
|
||||
"name": "AI Summarize Audio",
|
||||
"type": "n8n-nodes-base.httpRequest",
|
||||
"typeVersion": 4.2,
|
||||
"position": [
|
||||
448,
|
||||
304
|
||||
]
|
||||
},
|
||||
{
|
||||
"parameters": {
|
||||
"jsCode": "const llmResponse = $input.first().json;\nconst transcript = $('Whisper Transcribe 2').first().json;\nlet analysis;\n\ntry {\n analysis = JSON.parse(llmResponse.choices?.[0]?.message?.content || '{}');\n} catch(e) {\n analysis = { summary: llmResponse.choices?.[0]?.message?.content || 'Summary failed', key_points: [], action_items: [] };\n}\n\nreturn [{\n json: {\n transcript: transcript.text || '',\n duration_seconds: Math.round(transcript.duration || 0),\n language: transcript.language || 'en',\n summary: analysis.summary,\n key_points: analysis.key_points || [],\n action_items: analysis.action_items || []\n }\n}];"
|
||||
},
|
||||
"id": "cf41669b-8bf7-433b-b390-10765954b0a9",
|
||||
"name": "Format Summary Response",
|
||||
"type": "n8n-nodes-base.code",
|
||||
"typeVersion": 2,
|
||||
"position": [
|
||||
672,
|
||||
304
|
||||
]
|
||||
},
|
||||
{
|
||||
"parameters": {
|
||||
"jsCode": "const t = $input.first().json;\nconst mins = Math.floor(t.duration_seconds / 60);\nconst secs = t.duration_seconds % 60;\nreturn [{ json: {\n title: '\ud83c\udf99\ufe0f Transcription Complete',\n message: `Duration: ${mins}m ${secs}s \u00b7 ${t.segment_count} segments \u00b7 Language: ${t.language}`,\n priority: 3\n}}];\n"
|
||||
},
|
||||
"id": "5c692355-2a46-4c66-860a-c02a21c73597",
|
||||
"name": "Format Transcription Notify",
|
||||
"type": "n8n-nodes-base.code",
|
||||
"typeVersion": 2,
|
||||
"position": [
|
||||
672,
|
||||
0
|
||||
]
|
||||
},
|
||||
{
|
||||
"parameters": {
|
||||
"message": "={{ $json.message }}",
|
||||
"additionalFields": {
|
||||
"priority": "={{ $json.priority }}",
|
||||
"title": "={{ $json.title }}"
|
||||
},
|
||||
"options": {
|
||||
"contentType": "text/plain"
|
||||
}
|
||||
},
|
||||
"id": "0e96240d-50e2-49f7-be62-4f57451e0543",
|
||||
"name": "Notify Transcription Done",
|
||||
"type": "n8n-nodes-base.gotify",
|
||||
"typeVersion": 1,
|
||||
"position": [
|
||||
896,
|
||||
0
|
||||
],
|
||||
"credentials": {
|
||||
"gotifyApi": {
|
||||
"id": "SETUP_REQUIRED",
|
||||
"name": "Whisper"
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"parameters": {
|
||||
"jsCode": "const t = $('Format Transcript').first().json;\nconst title = `Transcript ${new Date().toLocaleDateString('en-US')}`;\nconst content = `Transcript\\nDate: ${new Date().toISOString()}\\nDuration: ${t.duration_seconds}s\\nLanguage: ${t.language}\\n\\n${t.timestamped_text || t.text}`;\nreturn [{ binary: { document: { data: Buffer.from(content,'utf-8').toString('base64'), mimeType:'text/plain', fileName: `${title}.txt` }}, json: { title } }];\n"
|
||||
},
|
||||
"id": "da9f4d8d-8c5b-413a-badd-71287dbe0a3c",
|
||||
"name": "Prepare Transcript Doc",
|
||||
"type": "n8n-nodes-base.code",
|
||||
"typeVersion": 2,
|
||||
"position": [
|
||||
1120,
|
||||
0
|
||||
]
|
||||
},
|
||||
{
|
||||
"parameters": {
|
||||
"method": "POST",
|
||||
"url": "https://paperless.paccoco.com/api/documents/post_document/",
|
||||
"options": {},
|
||||
"sendHeaders": true,
|
||||
"headerParameters": {
|
||||
"parameters": [
|
||||
{
|
||||
"name": "Authorization",
|
||||
"value": "Bearer ${PAPERLESS_API_TOKEN}"
|
||||
}
|
||||
]
|
||||
},
|
||||
"sendBody": true,
|
||||
"contentType": "multipart-form-data",
|
||||
"bodyParameters": {
|
||||
"parameters": [
|
||||
{
|
||||
"name": "document",
|
||||
"value": "data",
|
||||
"parameterType": "formBinaryData"
|
||||
},
|
||||
{
|
||||
"name": "title",
|
||||
"value": "={{ $json.title }}"
|
||||
},
|
||||
{
|
||||
"name": "tags[]",
|
||||
"value": "${PAPERLESS_TAG_TRANSCRIPTION}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
"id": "7c84da5a-41e2-4b38-9e58-19acfd73f81d",
|
||||
"name": "Save Transcript to Paperless",
|
||||
"type": "n8n-nodes-base.httpRequest",
|
||||
"typeVersion": 4.2,
|
||||
"position": [
|
||||
1344,
|
||||
0
|
||||
],
|
||||
"continueOnFail": true
|
||||
},
|
||||
{
|
||||
"parameters": {
|
||||
"jsCode": "const s = $input.first().json;\nconst mins = Math.floor(s.duration_seconds / 60);\nconst points = (s.key_points || []).slice(0,3).map((p,i) => `${i+1}. ${p}`).join('\\n');\nreturn [{ json: {\n title: '\ud83c\udf99\ufe0f Transcription + Summary Complete',\n message: `${mins}m audio\\n\\nSummary: ${s.summary}\\n\\nKey points:\\n${points}`,\n priority: 4\n}}];\n"
|
||||
},
|
||||
"id": "e55363e4-ff9b-4df4-b40a-fd23227c23e8",
|
||||
"name": "Format Summary Notify",
|
||||
"type": "n8n-nodes-base.code",
|
||||
"typeVersion": 2,
|
||||
"position": [
|
||||
896,
|
||||
304
|
||||
]
|
||||
},
|
||||
{
|
||||
"parameters": {
|
||||
"message": "={{ $json.message }}",
|
||||
"additionalFields": {
|
||||
"priority": "={{ $json.priority }}",
|
||||
"title": "={{ $json.title }}"
|
||||
},
|
||||
"options": {
|
||||
"contentType": "text/plain"
|
||||
}
|
||||
},
|
||||
"id": "b5defea9-35a0-4a22-8769-ed9496a022f6",
|
||||
"name": "Notify Summary Done",
|
||||
"type": "n8n-nodes-base.gotify",
|
||||
"typeVersion": 1,
|
||||
"position": [
|
||||
1120,
|
||||
304
|
||||
],
|
||||
"credentials": {
|
||||
"gotifyApi": {
|
||||
"id": "SETUP_REQUIRED",
|
||||
"name": "Whisper"
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"parameters": {
|
||||
"jsCode": "const s = $('Format Summary Response').first().json;\nconst title = `Audio Summary ${new Date().toLocaleDateString('en-US')}`;\nconst points = (s.key_points || []).map((p,i) => `${i+1}. ${p}`).join('\\n');\nconst actions = (s.action_items || []).map((a,i) => `${i+1}. ${a}`).join('\\n');\nconst content = [\n `Audio Transcription & Summary`, `Date: ${new Date().toISOString()}`,\n `Duration: ${Math.floor(s.duration_seconds/60)}m ${s.duration_seconds%60}s`,\n `Language: ${s.language}`, '', `Summary`, `=======`, s.summary,\n '', `Key Points`, `==========`, points,\n ...(actions ? ['', `Action Items`, `============`, actions] : []),\n '', `Full Transcript`, `==============`, s.transcript\n].join('\\n');\nreturn [{ binary: { document: { data: Buffer.from(content,'utf-8').toString('base64'), mimeType:'text/plain', fileName:`${title}.txt` }}, json: { title } }];\n"
|
||||
},
|
||||
"id": "1f31f608-7985-4a9a-afcf-6cf4e44150ba",
|
||||
"name": "Prepare Summary Doc",
|
||||
"type": "n8n-nodes-base.code",
|
||||
"typeVersion": 2,
|
||||
"position": [
|
||||
1344,
|
||||
304
|
||||
]
|
||||
},
|
||||
{
|
||||
"parameters": {
|
||||
"method": "POST",
|
||||
"url": "https://paperless.paccoco.com/api/documents/post_document/",
|
||||
"options": {},
|
||||
"sendHeaders": true,
|
||||
"headerParameters": {
|
||||
"parameters": [
|
||||
{
|
||||
"name": "Authorization",
|
||||
"value": "Bearer ${PAPERLESS_API_TOKEN}"
|
||||
}
|
||||
]
|
||||
},
|
||||
"sendBody": true,
|
||||
"contentType": "multipart-form-data",
|
||||
"bodyParameters": {
|
||||
"parameters": [
|
||||
{
|
||||
"name": "document",
|
||||
"value": "data",
|
||||
"parameterType": "formBinaryData"
|
||||
},
|
||||
{
|
||||
"name": "title",
|
||||
"value": "={{ $json.title }}"
|
||||
},
|
||||
{
|
||||
"name": "tags[]",
|
||||
"value": "${PAPERLESS_TAG_TRANSCRIPTION}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
"id": "e9eaee2f-d08e-4a76-af53-ed824ebfe68f",
|
||||
"name": "Save Summary to Paperless",
|
||||
"type": "n8n-nodes-base.httpRequest",
|
||||
"typeVersion": 4.2,
|
||||
"position": [
|
||||
1568,
|
||||
304
|
||||
],
|
||||
"continueOnFail": true
|
||||
}
|
||||
],
|
||||
"connections": {
|
||||
"Transcription Webhook": {
|
||||
"main": [
|
||||
[
|
||||
{
|
||||
"node": "Whisper Transcribe",
|
||||
"type": "main",
|
||||
"index": 0
|
||||
}
|
||||
]
|
||||
]
|
||||
},
|
||||
"Whisper Transcribe": {
|
||||
"main": [
|
||||
[
|
||||
{
|
||||
"node": "Format Transcript",
|
||||
"type": "main",
|
||||
"index": 0
|
||||
}
|
||||
]
|
||||
]
|
||||
},
|
||||
"Transcribe + Summarize Webhook": {
|
||||
"main": [
|
||||
[
|
||||
{
|
||||
"node": "Whisper Transcribe 2",
|
||||
"type": "main",
|
||||
"index": 0
|
||||
}
|
||||
]
|
||||
]
|
||||
},
|
||||
"Whisper Transcribe 2": {
|
||||
"main": [
|
||||
[
|
||||
{
|
||||
"node": "AI Summarize Audio",
|
||||
"type": "main",
|
||||
"index": 0
|
||||
}
|
||||
]
|
||||
]
|
||||
},
|
||||
"AI Summarize Audio": {
|
||||
"main": [
|
||||
[
|
||||
{
|
||||
"node": "Format Summary Response",
|
||||
"type": "main",
|
||||
"index": 0
|
||||
}
|
||||
]
|
||||
]
|
||||
},
|
||||
"Format Transcript": {
|
||||
"main": [
|
||||
[
|
||||
{
|
||||
"node": "Format Transcription Notify",
|
||||
"type": "main",
|
||||
"index": 0
|
||||
}
|
||||
]
|
||||
]
|
||||
},
|
||||
"Format Transcription Notify": {
|
||||
"main": [
|
||||
[
|
||||
{
|
||||
"node": "Notify Transcription Done",
|
||||
"type": "main",
|
||||
"index": 0
|
||||
}
|
||||
]
|
||||
]
|
||||
},
|
||||
"Notify Transcription Done": {
|
||||
"main": [
|
||||
[
|
||||
{
|
||||
"node": "Prepare Transcript Doc",
|
||||
"type": "main",
|
||||
"index": 0
|
||||
}
|
||||
]
|
||||
]
|
||||
},
|
||||
"Prepare Transcript Doc": {
|
||||
"main": [
|
||||
[
|
||||
{
|
||||
"node": "Save Transcript to Paperless",
|
||||
"type": "main",
|
||||
"index": 0
|
||||
}
|
||||
]
|
||||
]
|
||||
},
|
||||
"Format Summary Response": {
|
||||
"main": [
|
||||
[
|
||||
{
|
||||
"node": "Format Summary Notify",
|
||||
"type": "main",
|
||||
"index": 0
|
||||
}
|
||||
]
|
||||
]
|
||||
},
|
||||
"Format Summary Notify": {
|
||||
"main": [
|
||||
[
|
||||
{
|
||||
"node": "Notify Summary Done",
|
||||
"type": "main",
|
||||
"index": 0
|
||||
}
|
||||
]
|
||||
]
|
||||
},
|
||||
"Notify Summary Done": {
|
||||
"main": [
|
||||
[
|
||||
{
|
||||
"node": "Prepare Summary Doc",
|
||||
"type": "main",
|
||||
"index": 0
|
||||
}
|
||||
]
|
||||
]
|
||||
},
|
||||
"Prepare Summary Doc": {
|
||||
"main": [
|
||||
[
|
||||
{
|
||||
"node": "Save Summary to Paperless",
|
||||
"type": "main",
|
||||
"index": 0
|
||||
}
|
||||
]
|
||||
]
|
||||
}
|
||||
},
|
||||
"settings": {
|
||||
"executionOrder": "v1",
|
||||
"binaryMode": "separate"
|
||||
},
|
||||
"staticData": null,
|
||||
"meta": null,
|
||||
"pinData": {},
|
||||
"versionId": "20275daf-b924-4311-9a7f-116a99911764",
|
||||
"activeVersionId": "20275daf-b924-4311-9a7f-116a99911764",
|
||||
"versionCounter": 12,
|
||||
"triggerCount": 2,
|
||||
"tags": [
|
||||
{
|
||||
"updatedAt": "2026-05-06T22:39:59.719Z",
|
||||
"createdAt": "2026-05-06T22:39:59.719Z",
|
||||
"id": "GLLrEIR8I7eA4ydv",
|
||||
"name": "transcription"
|
||||
},
|
||||
{
|
||||
"updatedAt": "2026-05-06T22:39:59.716Z",
|
||||
"createdAt": "2026-05-06T22:39:59.716Z",
|
||||
"id": "tuPxDoKFqpVrrQoX",
|
||||
"name": "whisper"
|
||||
},
|
||||
{
|
||||
"updatedAt": "2026-05-06T22:38:51.678Z",
|
||||
"createdAt": "2026-05-06T22:38:51.678Z",
|
||||
"id": "S9FxzVfHLsHmyzyt",
|
||||
"name": "ai"
|
||||
}
|
||||
],
|
||||
"shared": [
|
||||
{
|
||||
"updatedAt": "2026-05-06T23:30:36.237Z",
|
||||
"createdAt": "2026-05-06T23:30:36.237Z",
|
||||
"role": "workflow:owner",
|
||||
"workflowId": "QEVnqyN4h5kcpVuJ",
|
||||
"projectId": "dxCRnBdX5uJizCGa",
|
||||
"project": {
|
||||
"updatedAt": "2026-05-06T01:10:37.484Z",
|
||||
"createdAt": "2026-05-06T01:08:07.721Z",
|
||||
"id": "dxCRnBdX5uJizCGa",
|
||||
"name": "Wilfred Fizzlepoof <admin@paccoco.com>",
|
||||
"type": "personal",
|
||||
"icon": null,
|
||||
"description": null,
|
||||
"creatorId": "47b2227f-2fc2-4a08-bd35-ea157b92df0d"
|
||||
}
|
||||
}
|
||||
],
|
||||
"versionMetadata": {
|
||||
"name": "Version 20275daf",
|
||||
"description": ""
|
||||
}
|
||||
}
|
||||
357
n8n-workflows/05-paperless-to-rag.json
Normal file
357
n8n-workflows/05-paperless-to-rag.json
Normal file
@@ -0,0 +1,357 @@
|
||||
{
|
||||
"updatedAt": "2026-05-07T00:48:49.355Z",
|
||||
"createdAt": "2026-05-07T00:48:49.355Z",
|
||||
"id": "4YKnRQ9rOiQCeYEU",
|
||||
"name": "Paperless \u2192 RAG Ingest",
|
||||
"description": null,
|
||||
"active": true,
|
||||
"isArchived": false,
|
||||
"nodes": [
|
||||
{
|
||||
"parameters": {
|
||||
"httpMethod": "POST",
|
||||
"path": "paperless/rag-ingest",
|
||||
"options": {}
|
||||
},
|
||||
"id": "a86d7025-bbf0-4c1b-a59c-ad254ddc913e",
|
||||
"name": "Paperless RAG Webhook",
|
||||
"type": "n8n-nodes-base.webhook",
|
||||
"typeVersion": 2,
|
||||
"position": [
|
||||
0,
|
||||
0
|
||||
],
|
||||
"webhookId": "f2eccecb-9935-425b-be6a-f57fe4473f34"
|
||||
},
|
||||
{
|
||||
"parameters": {
|
||||
"jsCode": "const data = $input.first().json;\nconst body = data.body || data;\n\nlet docId = body.document_id || body.id || body.doc_id || body.pk;\n\nif (!docId || typeof docId === 'string' && docId.includes('{')) {\n docId = null;\n}\n\nreturn [{ json: { docId } }];"
|
||||
},
|
||||
"id": "8f6af849-bdd9-466e-af81-84d057fd7ca1",
|
||||
"name": "Extract Document ID",
|
||||
"type": "n8n-nodes-base.code",
|
||||
"typeVersion": 2,
|
||||
"position": [
|
||||
224,
|
||||
0
|
||||
]
|
||||
},
|
||||
{
|
||||
"parameters": {
|
||||
"url": "=https://paperless.paccoco.com/api/documents/{{ $json.docId ? $json.docId + '/' : '?ordering=-added&page_size=1' }}",
|
||||
"sendHeaders": true,
|
||||
"headerParameters": {
|
||||
"parameters": [
|
||||
{
|
||||
"name": "Authorization",
|
||||
"value": "Token 4fb34ba35b4ab76a0715e8c56faf00a31de4fb5d"
|
||||
}
|
||||
]
|
||||
},
|
||||
"options": {}
|
||||
},
|
||||
"id": "0b53d951-40cd-4ef0-ab81-3dea73f8b1d7",
|
||||
"name": "Fetch Document",
|
||||
"type": "n8n-nodes-base.httpRequest",
|
||||
"typeVersion": 4.2,
|
||||
"position": [
|
||||
448,
|
||||
0
|
||||
]
|
||||
},
|
||||
{
|
||||
"parameters": {
|
||||
"jsCode": "const response = $input.first().json;\nconst doc = response.results ? response.results[0] : response;\n\nif (!doc || !doc.id) {\n throw new Error('No document found in response');\n}\n\nconst content = doc.content || '';\nconst title = doc.title || 'Untitled';\nconst correspondent = doc.correspondent_name || 'Unknown';\nconst created = doc.created || new Date().toISOString();\nconst docType = doc.document_type_name || 'unknown';\nconst tags = (doc.tags || []).join(', ');\n\n// Build rich text for RAG with metadata context\nconst ragText = `Document: ${title}\\nCorrespondent: ${correspondent}\\nType: ${docType}\\nTags: ${tags}\\nDate: ${created}\\n\\n${content}`;\n\nreturn [{\n json: {\n documentId: doc.id,\n title,\n ragText,\n source: `paperless-doc-${doc.id}`,\n metadata: {\n paperless_id: doc.id,\n title,\n correspondent,\n document_type: docType,\n created\n }\n }\n}];"
|
||||
},
|
||||
"id": "e3d506bc-1677-4825-b3d3-f6187156da22",
|
||||
"name": "Prepare RAG Text",
|
||||
"type": "n8n-nodes-base.code",
|
||||
"typeVersion": 2,
|
||||
"position": [
|
||||
672,
|
||||
0
|
||||
]
|
||||
},
|
||||
{
|
||||
"parameters": {
|
||||
"jsCode": "// Split text into chunks for embedding\nconst input = $input.first().json;\nconst text = input.ragText || '';\nconst source = input.source;\nconst metadata = input.metadata;\n\nconst CHUNK_SIZE = 500;\nconst OVERLAP = 50;\n\nconst chunks = [];\nconst sentences = text.split(/(?<=[.!?])\\s+/);\nlet current = '';\n\nfor (const sentence of sentences) {\n if ((current + ' ' + sentence).length > CHUNK_SIZE && current.length > 0) {\n chunks.push(current.trim());\n const words = current.split(' ');\n current = words.slice(-OVERLAP).join(' ') + ' ' + sentence;\n } else {\n current = current ? current + ' ' + sentence : sentence;\n }\n}\nif (current.trim()) chunks.push(current.trim());\n\nreturn chunks.map((chunk, i) => ({\n json: {\n chunk,\n chunkIndex: i,\n totalChunks: chunks.length,\n source,\n metadata\n }\n}));"
|
||||
},
|
||||
"id": "9a523501-3adb-41a3-b963-b6bb5adb25fe",
|
||||
"name": "Chunk Text",
|
||||
"type": "n8n-nodes-base.code",
|
||||
"typeVersion": 2,
|
||||
"position": [
|
||||
880,
|
||||
0
|
||||
]
|
||||
},
|
||||
{
|
||||
"parameters": {
|
||||
"method": "POST",
|
||||
"url": "http://10.5.1.6:11434/v1/embeddings",
|
||||
"sendHeaders": true,
|
||||
"headerParameters": {
|
||||
"parameters": [
|
||||
{
|
||||
"name": "Content-Type",
|
||||
"value": "application/json"
|
||||
}
|
||||
]
|
||||
},
|
||||
"sendBody": true,
|
||||
"specifyBody": "json",
|
||||
"jsonBody": "={\n \"model\": \"nomic-embed-text\",\n \"input\": \"{{ $json.chunk }}\"\n}",
|
||||
"options": {
|
||||
"batching": {
|
||||
"batch": {
|
||||
"batchSize": 5,
|
||||
"batchInterval": 500
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"id": "0b631ea4-b11f-4533-960f-f5161ac798f3",
|
||||
"name": "Get Embedding",
|
||||
"type": "n8n-nodes-base.httpRequest",
|
||||
"typeVersion": 4.2,
|
||||
"position": [
|
||||
1104,
|
||||
0
|
||||
]
|
||||
},
|
||||
{
|
||||
"parameters": {
|
||||
"jsCode": "const embeddingResponse = $input.first().json;\nconst chunkData = $('Chunk Text').item;\nconst embedding = embeddingResponse.data?.[0]?.embedding || [];\n\nconst pointId = 'xxxxxxxx-xxxx-4xxx-yxxx-xxxxxxxxxxxx'.replace(/[xy]/g, c => {\n const r = Math.random() * 16 | 0;\n return (c === 'x' ? r : (r & 0x3 | 0x8)).toString(16);\n});\n\nreturn [{\n json: {\n id: pointId,\n vector: embedding,\n payload: {\n text: chunkData.json.chunk,\n source: chunkData.json.source,\n chunkIndex: chunkData.json.chunkIndex,\n totalChunks: chunkData.json.totalChunks,\n metadata: chunkData.json.metadata,\n ingested_at: new Date().toISOString()\n }\n }\n}];"
|
||||
},
|
||||
"id": "ad9dc677-e3a2-464e-a82e-2aa4d8b9b042",
|
||||
"name": "Prepare Qdrant Point",
|
||||
"type": "n8n-nodes-base.code",
|
||||
"typeVersion": 2,
|
||||
"position": [
|
||||
1328,
|
||||
0
|
||||
]
|
||||
},
|
||||
{
|
||||
"parameters": {
|
||||
"method": "PUT",
|
||||
"url": "http://10.5.1.6:6333/collections/knowledge_base/points",
|
||||
"sendHeaders": true,
|
||||
"headerParameters": {
|
||||
"parameters": [
|
||||
{
|
||||
"name": "Content-Type",
|
||||
"value": "application/json"
|
||||
}
|
||||
]
|
||||
},
|
||||
"sendBody": true,
|
||||
"specifyBody": "json",
|
||||
"jsonBody": "={\n \"points\": [\n {\n \"id\": \"{{ $json.id }}\",\n \"vector\": {{ JSON.stringify($json.vector) }},\n \"payload\": {{ JSON.stringify($json.payload) }}\n }\n ]\n}",
|
||||
"options": {}
|
||||
},
|
||||
"id": "e977b536-d470-4da0-a970-794a858e8aa9",
|
||||
"name": "Upsert to Qdrant",
|
||||
"type": "n8n-nodes-base.httpRequest",
|
||||
"typeVersion": 4.2,
|
||||
"position": [
|
||||
1552,
|
||||
0
|
||||
]
|
||||
},
|
||||
{
|
||||
"parameters": {
|
||||
"jsCode": "const ragText = $('Prepare RAG Text').first().json;\nconst chunks = $('Chunk Text').first().json;\nreturn [{\n json: {\n title: '\ud83d\udcda Document Added to RAG',\n message: `${ragText.title} has been chunked and ingested into the knowledge base (${chunks.totalChunks || '?'} chunks)`,\n priority: 3\n }\n}];"
|
||||
},
|
||||
"id": "fmt05-0001-0000-0000-000000000005",
|
||||
"name": "Format Gotify Message",
|
||||
"type": "n8n-nodes-base.code",
|
||||
"typeVersion": 2,
|
||||
"position": [
|
||||
1520,
|
||||
0
|
||||
]
|
||||
},
|
||||
{
|
||||
"parameters": {
|
||||
"message": "={{ $json.message }}",
|
||||
"additionalFields": {
|
||||
"priority": "={{ $json.priority }}",
|
||||
"title": "={{ $json.title }}"
|
||||
},
|
||||
"options": {
|
||||
"contentType": "text/plain"
|
||||
}
|
||||
},
|
||||
"id": "ccdee208-b8f8-4ad4-a20c-94c4735d2e3d",
|
||||
"name": "Notify via Gotify",
|
||||
"type": "n8n-nodes-base.gotify",
|
||||
"typeVersion": 1,
|
||||
"position": [
|
||||
1760,
|
||||
0
|
||||
],
|
||||
"credentials": {
|
||||
"gotifyApi": {
|
||||
"id": "SETUP_REQUIRED",
|
||||
"name": "Paperless RAG"
|
||||
}
|
||||
}
|
||||
}
|
||||
],
|
||||
"connections": {
|
||||
"Paperless RAG Webhook": {
|
||||
"main": [
|
||||
[
|
||||
{
|
||||
"node": "Extract Document ID",
|
||||
"type": "main",
|
||||
"index": 0
|
||||
}
|
||||
]
|
||||
]
|
||||
},
|
||||
"Extract Document ID": {
|
||||
"main": [
|
||||
[
|
||||
{
|
||||
"node": "Fetch Document",
|
||||
"type": "main",
|
||||
"index": 0
|
||||
}
|
||||
]
|
||||
]
|
||||
},
|
||||
"Fetch Document": {
|
||||
"main": [
|
||||
[
|
||||
{
|
||||
"node": "Prepare RAG Text",
|
||||
"type": "main",
|
||||
"index": 0
|
||||
}
|
||||
]
|
||||
]
|
||||
},
|
||||
"Prepare RAG Text": {
|
||||
"main": [
|
||||
[
|
||||
{
|
||||
"node": "Chunk Text",
|
||||
"type": "main",
|
||||
"index": 0
|
||||
}
|
||||
]
|
||||
]
|
||||
},
|
||||
"Chunk Text": {
|
||||
"main": [
|
||||
[
|
||||
{
|
||||
"node": "Get Embedding",
|
||||
"type": "main",
|
||||
"index": 0
|
||||
}
|
||||
]
|
||||
]
|
||||
},
|
||||
"Get Embedding": {
|
||||
"main": [
|
||||
[
|
||||
{
|
||||
"node": "Prepare Qdrant Point",
|
||||
"type": "main",
|
||||
"index": 0
|
||||
}
|
||||
]
|
||||
]
|
||||
},
|
||||
"Prepare Qdrant Point": {
|
||||
"main": [
|
||||
[
|
||||
{
|
||||
"node": "Upsert to Qdrant",
|
||||
"type": "main",
|
||||
"index": 0
|
||||
}
|
||||
]
|
||||
]
|
||||
},
|
||||
"Upsert to Qdrant": {
|
||||
"main": [
|
||||
[
|
||||
{
|
||||
"node": "Format Gotify Message",
|
||||
"type": "main",
|
||||
"index": 0
|
||||
}
|
||||
]
|
||||
]
|
||||
},
|
||||
"Format Gotify Message": {
|
||||
"main": [
|
||||
[
|
||||
{
|
||||
"node": "Notify via Gotify",
|
||||
"type": "main",
|
||||
"index": 0
|
||||
}
|
||||
]
|
||||
]
|
||||
}
|
||||
},
|
||||
"settings": {
|
||||
"executionOrder": "v1",
|
||||
"binaryMode": "separate"
|
||||
},
|
||||
"staticData": null,
|
||||
"meta": null,
|
||||
"pinData": {},
|
||||
"versionId": "3c20b423-8708-4540-8bf9-8db4880335a5",
|
||||
"activeVersionId": "3c20b423-8708-4540-8bf9-8db4880335a5",
|
||||
"versionCounter": 12,
|
||||
"triggerCount": 1,
|
||||
"tags": [
|
||||
{
|
||||
"updatedAt": "2026-05-06T22:39:46.476Z",
|
||||
"createdAt": "2026-05-06T22:39:46.476Z",
|
||||
"id": "NxHjuO4MMd5kdPR7",
|
||||
"name": "paperless"
|
||||
},
|
||||
{
|
||||
"updatedAt": "2026-05-06T22:39:33.284Z",
|
||||
"createdAt": "2026-05-06T22:39:33.284Z",
|
||||
"id": "7uuEyQrIlcuruZBk",
|
||||
"name": "rag"
|
||||
},
|
||||
{
|
||||
"updatedAt": "2026-05-06T22:38:51.678Z",
|
||||
"createdAt": "2026-05-06T22:38:51.678Z",
|
||||
"id": "S9FxzVfHLsHmyzyt",
|
||||
"name": "ai"
|
||||
}
|
||||
],
|
||||
"shared": [
|
||||
{
|
||||
"updatedAt": "2026-05-07T00:48:49.355Z",
|
||||
"createdAt": "2026-05-07T00:48:49.355Z",
|
||||
"role": "workflow:owner",
|
||||
"workflowId": "4YKnRQ9rOiQCeYEU",
|
||||
"projectId": "dxCRnBdX5uJizCGa",
|
||||
"project": {
|
||||
"updatedAt": "2026-05-06T01:10:37.484Z",
|
||||
"createdAt": "2026-05-06T01:08:07.721Z",
|
||||
"id": "dxCRnBdX5uJizCGa",
|
||||
"name": "Wilfred Fizzlepoof <admin@paccoco.com>",
|
||||
"type": "personal",
|
||||
"icon": null,
|
||||
"description": null,
|
||||
"creatorId": "47b2227f-2fc2-4a08-bd35-ea157b92df0d"
|
||||
}
|
||||
}
|
||||
],
|
||||
"versionMetadata": {
|
||||
"name": "Version 3c20b423",
|
||||
"description": ""
|
||||
}
|
||||
}
|
||||
588
n8n-workflows/06-web-scrape-summarize.json
Normal file
588
n8n-workflows/06-web-scrape-summarize.json
Normal file
File diff suppressed because one or more lines are too long
210
n8n-workflows/07-git-commit-summarizer.json
Normal file
210
n8n-workflows/07-git-commit-summarizer.json
Normal file
@@ -0,0 +1,210 @@
|
||||
{
|
||||
"updatedAt": "2026-05-07T00:49:43.226Z",
|
||||
"createdAt": "2026-05-07T00:49:43.226Z",
|
||||
"id": "CFYWbBRx9RPf4HjS",
|
||||
"name": "Git Commit \u2192 AI Summary \u2192 Gotify",
|
||||
"description": null,
|
||||
"active": true,
|
||||
"isArchived": false,
|
||||
"nodes": [
|
||||
{
|
||||
"parameters": {
|
||||
"httpMethod": "POST",
|
||||
"path": "git/push",
|
||||
"options": {}
|
||||
},
|
||||
"id": "fe377ac9-adcf-427d-a096-679f5469f8d8",
|
||||
"name": "Git Push Webhook",
|
||||
"type": "n8n-nodes-base.webhook",
|
||||
"typeVersion": 2,
|
||||
"position": [
|
||||
0,
|
||||
0
|
||||
],
|
||||
"webhookId": "d1d31942-6fe8-48a2-aab7-7eefaf800420"
|
||||
},
|
||||
{
|
||||
"parameters": {
|
||||
"jsCode": "const data = $input.first().json;\nconst body = data.body || data;\n\n// Works with Gitea, Forgejo, GitHub, and GitLab webhook payloads\nconst repo = body.repository?.full_name || body.repository?.name || body.project?.path_with_namespace || 'unknown-repo';\nconst branch = (body.ref || '').replace('refs/heads/', '');\nconst pusher = body.pusher?.name || body.pusher?.login || body.user_name || body.sender?.login || 'unknown';\n\n// Extract commits\nconst commits = (body.commits || []).map(c => ({\n id: (c.id || c.sha || '').substring(0, 7),\n message: (c.message || '').split('\\n')[0],\n author: c.author?.name || c.author?.username || 'unknown',\n added: (c.added || []).length,\n modified: (c.modified || []).length,\n removed: (c.removed || []).length\n}));\n\nconst commitSummary = commits.map(c =>\n `${c.id}: ${c.message} (by ${c.author}) [+${c.added} ~${c.modified} -${c.removed}]`\n).join('\\n');\n\nreturn [{\n json: {\n repo,\n branch,\n pusher,\n commitCount: commits.length,\n commitSummary,\n compareUrl: body.compare_url || body.compare || ''\n }\n}];"
|
||||
},
|
||||
"id": "8dda4f14-9563-490f-89b3-aef80f64ba0b",
|
||||
"name": "Parse Git Payload",
|
||||
"type": "n8n-nodes-base.code",
|
||||
"typeVersion": 2,
|
||||
"position": [
|
||||
224,
|
||||
0
|
||||
]
|
||||
},
|
||||
{
|
||||
"parameters": {
|
||||
"method": "POST",
|
||||
"url": "http://10.5.1.6:4000/v1/chat/completions",
|
||||
"sendHeaders": true,
|
||||
"headerParameters": {
|
||||
"parameters": [
|
||||
{
|
||||
"name": "Content-Type",
|
||||
"value": "application/json"
|
||||
},
|
||||
{
|
||||
"name": "Authorization",
|
||||
"value": "Bearer sk-REPLACE_WITH_LITELLM_KEY"
|
||||
}
|
||||
]
|
||||
},
|
||||
"sendBody": true,
|
||||
"specifyBody": "json",
|
||||
"jsonBody": "={\n \"model\": \"light\",\n \"messages\": [\n {\n \"role\": \"system\",\n \"content\": \"You are a git commit summarizer. Given a list of commits pushed to a repo, write a 1-2 sentence summary of what changed. Be concise and technical. Focus on the purpose of the changes, not just listing files.\"\n },\n {\n \"role\": \"user\",\n \"content\": \"Repository: {{ $json.repo }}\\nBranch: {{ $json.branch }}\\nPushed by: {{ $json.pusher }}\\n\\nCommits ({{ $json.commitCount }}):\\n{{ $json.commitSummary }}\"\n }\n ],\n \"max_tokens\": 150,\n \"temperature\": 0.2\n}",
|
||||
"options": {}
|
||||
},
|
||||
"id": "c6b2b71b-2612-4ef7-8561-8adccbfa9be8",
|
||||
"name": "AI Summarize Commits",
|
||||
"type": "n8n-nodes-base.httpRequest",
|
||||
"typeVersion": 4.2,
|
||||
"position": [
|
||||
448,
|
||||
0
|
||||
]
|
||||
},
|
||||
{
|
||||
"parameters": {
|
||||
"jsCode": "const response = $input.first().json;\nconst gitData = $('Parse Git Payload').first().json;\nconst summary = response.choices?.[0]?.message?.content || 'Could not summarize';\n\nconst title = `\ud83d\udd00 ${gitData.repo}:${gitData.branch} (${gitData.commitCount} commit${gitData.commitCount !== 1 ? 's' : ''})`;\n\nreturn [{\n json: {\n title,\n message: `${summary}\\n\\nPushed by: ${gitData.pusher}`,\n priority: 3\n }\n}];"
|
||||
},
|
||||
"id": "94f5c120-0ae6-4a23-9556-d4f564f9f967",
|
||||
"name": "Format Notification",
|
||||
"type": "n8n-nodes-base.code",
|
||||
"typeVersion": 2,
|
||||
"position": [
|
||||
672,
|
||||
0
|
||||
]
|
||||
},
|
||||
{
|
||||
"parameters": {
|
||||
"message": "={{ $json.message }}",
|
||||
"additionalFields": {
|
||||
"priority": "={{ $json.priority }}",
|
||||
"title": "={{ $json.title }}"
|
||||
},
|
||||
"options": {
|
||||
"contentType": "text/plain"
|
||||
}
|
||||
},
|
||||
"id": "4f731683-574c-4a39-84fe-025280193ba1",
|
||||
"name": "Push to Gotify",
|
||||
"type": "n8n-nodes-base.gotify",
|
||||
"typeVersion": 1,
|
||||
"position": [
|
||||
880,
|
||||
0
|
||||
],
|
||||
"credentials": {
|
||||
"gotifyApi": {
|
||||
"id": "SETUP_REQUIRED",
|
||||
"name": "Git Commits"
|
||||
}
|
||||
}
|
||||
}
|
||||
],
|
||||
"connections": {
|
||||
"Git Push Webhook": {
|
||||
"main": [
|
||||
[
|
||||
{
|
||||
"node": "Parse Git Payload",
|
||||
"type": "main",
|
||||
"index": 0
|
||||
}
|
||||
]
|
||||
]
|
||||
},
|
||||
"Parse Git Payload": {
|
||||
"main": [
|
||||
[
|
||||
{
|
||||
"node": "AI Summarize Commits",
|
||||
"type": "main",
|
||||
"index": 0
|
||||
}
|
||||
]
|
||||
]
|
||||
},
|
||||
"AI Summarize Commits": {
|
||||
"main": [
|
||||
[
|
||||
{
|
||||
"node": "Format Notification",
|
||||
"type": "main",
|
||||
"index": 0
|
||||
}
|
||||
]
|
||||
]
|
||||
},
|
||||
"Format Notification": {
|
||||
"main": [
|
||||
[
|
||||
{
|
||||
"node": "Push to Gotify",
|
||||
"type": "main",
|
||||
"index": 0
|
||||
}
|
||||
]
|
||||
]
|
||||
}
|
||||
},
|
||||
"settings": {
|
||||
"executionOrder": "v1",
|
||||
"binaryMode": "separate"
|
||||
},
|
||||
"staticData": null,
|
||||
"meta": null,
|
||||
"pinData": {},
|
||||
"versionId": "61fca126-b5dd-4425-b0ee-c621d3ed58af",
|
||||
"activeVersionId": "61fca126-b5dd-4425-b0ee-c621d3ed58af",
|
||||
"versionCounter": 12,
|
||||
"triggerCount": 1,
|
||||
"tags": [
|
||||
{
|
||||
"updatedAt": "2026-05-07T00:49:41.640Z",
|
||||
"createdAt": "2026-05-07T00:49:41.640Z",
|
||||
"id": "36CVfQVO41QQyOmh",
|
||||
"name": "notifications"
|
||||
},
|
||||
{
|
||||
"updatedAt": "2026-05-07T00:49:41.638Z",
|
||||
"createdAt": "2026-05-07T00:49:41.638Z",
|
||||
"id": "2oumYE52ygvhrieZ",
|
||||
"name": "git"
|
||||
},
|
||||
{
|
||||
"updatedAt": "2026-05-06T22:38:51.678Z",
|
||||
"createdAt": "2026-05-06T22:38:51.678Z",
|
||||
"id": "S9FxzVfHLsHmyzyt",
|
||||
"name": "ai"
|
||||
}
|
||||
],
|
||||
"shared": [
|
||||
{
|
||||
"updatedAt": "2026-05-07T00:49:43.226Z",
|
||||
"createdAt": "2026-05-07T00:49:43.226Z",
|
||||
"role": "workflow:owner",
|
||||
"workflowId": "CFYWbBRx9RPf4HjS",
|
||||
"projectId": "dxCRnBdX5uJizCGa",
|
||||
"project": {
|
||||
"updatedAt": "2026-05-06T01:10:37.484Z",
|
||||
"createdAt": "2026-05-06T01:08:07.721Z",
|
||||
"id": "dxCRnBdX5uJizCGa",
|
||||
"name": "Wilfred Fizzlepoof <admin@paccoco.com>",
|
||||
"type": "personal",
|
||||
"icon": null,
|
||||
"description": null,
|
||||
"creatorId": "47b2227f-2fc2-4a08-bd35-ea157b92df0d"
|
||||
}
|
||||
}
|
||||
],
|
||||
"versionMetadata": {
|
||||
"name": "Version 61fca126",
|
||||
"description": ""
|
||||
}
|
||||
}
|
||||
532
n8n-workflows/08-class-recording-rag.json
Normal file
532
n8n-workflows/08-class-recording-rag.json
Normal file
@@ -0,0 +1,532 @@
|
||||
{
|
||||
"updatedAt": "2026-05-07T01:16:50.340Z",
|
||||
"createdAt": "2026-05-07T01:16:50.340Z",
|
||||
"id": "91I278E72FAaQenD",
|
||||
"name": "Class Recording \u2192 Transcribe \u2192 RAG",
|
||||
"description": null,
|
||||
"active": true,
|
||||
"isArchived": false,
|
||||
"nodes": [
|
||||
{
|
||||
"parameters": {
|
||||
"httpMethod": "POST",
|
||||
"path": "class/upload",
|
||||
"responseMode": "lastNode",
|
||||
"responseData": "allEntries",
|
||||
"options": {
|
||||
"rawBody": true
|
||||
}
|
||||
},
|
||||
"id": "e475f17b-8560-4376-852e-a613ba6b300e",
|
||||
"name": "Class Upload Webhook",
|
||||
"type": "n8n-nodes-base.webhook",
|
||||
"typeVersion": 2,
|
||||
"position": [
|
||||
0,
|
||||
0
|
||||
],
|
||||
"webhookId": "d0ee8226-db56-477e-8a3f-572a9129aea9"
|
||||
},
|
||||
{
|
||||
"parameters": {
|
||||
"jsCode": "const data = $input.first().json;\nconst body = data.body || data;\n\n// Class name comes from:\n// 1. Form field 'class_name' in multipart upload\n// 2. Query parameter ?class=CS101\n// 3. Header X-Class-Name\nconst className = body.class_name || data.headers?.['x-class-name'] || data.query?.class || 'unclassified';\n\n// Lecture title from:\n// 1. Form field 'lecture_title'\n// 2. Query param ?title=...\n// 3. Header X-Lecture-Title\n// 4. Form field 'filename' (original filename)\nconst rawTitle = body.lecture_title || data.headers?.['x-lecture-title'] || data.query?.title || body.filename || 'Untitled Lecture';\nconst lectureTitle = rawTitle.replace(/\\.wav$/i, '').replace(/[-_]/g, ' ');\n\nconst classSlug = className.toLowerCase().replace(/[^a-z0-9]+/g, '-').replace(/^-|-$/g, '');\n\nreturn [{\n json: {\n className,\n classSlug,\n lectureTitle,\n uploadedAt: new Date().toISOString()\n }\n}];"
|
||||
},
|
||||
"id": "24e4a726-ab50-4767-a144-1751616c781d",
|
||||
"name": "Extract Class Info",
|
||||
"type": "n8n-nodes-base.code",
|
||||
"typeVersion": 2,
|
||||
"position": [
|
||||
224,
|
||||
0
|
||||
]
|
||||
},
|
||||
{
|
||||
"parameters": {
|
||||
"method": "POST",
|
||||
"url": "http://10.5.1.6:8786/v1/audio/transcriptions",
|
||||
"sendHeaders": true,
|
||||
"headerParameters": {
|
||||
"parameters": [
|
||||
{
|
||||
"name": "Accept",
|
||||
"value": "application/json"
|
||||
}
|
||||
]
|
||||
},
|
||||
"sendBody": true,
|
||||
"contentType": "multipart-form-data",
|
||||
"bodyParameters": {
|
||||
"parameters": [
|
||||
{
|
||||
"parameterType": "formBinaryData",
|
||||
"name": "file",
|
||||
"inputDataFieldName": "data"
|
||||
},
|
||||
{
|
||||
"name": "model",
|
||||
"value": "Systran/faster-whisper-large-v3"
|
||||
},
|
||||
{
|
||||
"name": "language",
|
||||
"value": "en"
|
||||
},
|
||||
{
|
||||
"name": "response_format",
|
||||
"value": "verbose_json"
|
||||
}
|
||||
]
|
||||
},
|
||||
"options": {
|
||||
"timeout": 600000
|
||||
}
|
||||
},
|
||||
"id": "fcd1062d-8f60-4214-85ee-f53b914abcbb",
|
||||
"name": "Whisper Transcribe",
|
||||
"type": "n8n-nodes-base.httpRequest",
|
||||
"typeVersion": 4.2,
|
||||
"position": [
|
||||
448,
|
||||
0
|
||||
]
|
||||
},
|
||||
{
|
||||
"parameters": {
|
||||
"jsCode": "const result = $input.first().json;\nconst classInfo = $('Extract Class Info').first().json;\n\nconst segments = result.segments || [];\nconst fullText = result.text || segments.map(s => s.text).join(' ');\nconst duration = result.duration || segments[segments.length - 1]?.end || 0;\n\nconst timestamped = segments.map(s => {\n const mins = Math.floor(s.start / 60);\n const secs = Math.floor(s.start % 60);\n return `[${mins}:${String(secs).padStart(2, '0')}] ${s.text.trim()}`;\n}).join('\\n');\n\nreturn [{\n json: {\n className: classInfo.className,\n classSlug: classInfo.classSlug,\n lectureTitle: classInfo.lectureTitle,\n fullText: fullText.trim(),\n timestampedText: timestamped,\n durationMinutes: Math.round(duration / 60),\n segmentCount: segments.length\n }\n}];"
|
||||
},
|
||||
"id": "ecf09ad2-327e-4a82-90f3-95da5590e0ac",
|
||||
"name": "Format Transcript",
|
||||
"type": "n8n-nodes-base.code",
|
||||
"typeVersion": 2,
|
||||
"position": [
|
||||
672,
|
||||
0
|
||||
]
|
||||
},
|
||||
{
|
||||
"parameters": {
|
||||
"method": "POST",
|
||||
"url": "http://10.5.1.6:4000/v1/chat/completions",
|
||||
"sendHeaders": true,
|
||||
"headerParameters": {
|
||||
"parameters": [
|
||||
{
|
||||
"name": "Content-Type",
|
||||
"value": "application/json"
|
||||
},
|
||||
{
|
||||
"name": "Authorization",
|
||||
"value": "Bearer sk-REPLACE_WITH_LITELLM_KEY"
|
||||
}
|
||||
]
|
||||
},
|
||||
"sendBody": true,
|
||||
"specifyBody": "json",
|
||||
"jsonBody": "={\n \"model\": \"medium\",\n \"messages\": [\n {\n \"role\": \"system\",\n \"content\": \"You are an academic note-taking assistant. Given a class lecture transcription, provide:\\n1. A concise summary (3-5 sentences)\\n2. Key concepts covered (as a list)\\n3. Important definitions or formulas\\n4. Any assignments or deadlines mentioned\\n\\nRespond in JSON with keys: summary, key_concepts (array), definitions (array of {term, definition}), assignments (array)\"\n },\n {\n \"role\": \"user\",\n \"content\": \"Class: {{ $json.className }}\\nLecture: {{ $json.lectureTitle }}\\n\\nTranscription (first 3000 chars):\\n{{ $json.fullText.substring(0, 3000) }}\"\n }\n ],\n \"max_tokens\": 600,\n \"temperature\": 0.2,\n \"response_format\": { \"type\": \"json_object\" }\n}",
|
||||
"options": {}
|
||||
},
|
||||
"id": "e7407d06-f7f4-4105-a817-055d4703d59d",
|
||||
"name": "AI Extract Notes",
|
||||
"type": "n8n-nodes-base.httpRequest",
|
||||
"typeVersion": 4.2,
|
||||
"position": [
|
||||
880,
|
||||
0
|
||||
]
|
||||
},
|
||||
{
|
||||
"parameters": {
|
||||
"jsCode": "const llmResponse = $input.first().json;\nconst transcriptData = $('Format Transcript').first().json;\n\nlet notes;\ntry {\n notes = JSON.parse(llmResponse.choices?.[0]?.message?.content || '{}');\n} catch(e) {\n notes = { summary: 'Could not parse notes', key_concepts: [], definitions: [], assignments: [] };\n}\n\nconst ragText = [\n `Class: ${transcriptData.className}`,\n `Lecture: ${transcriptData.lectureTitle}`,\n `Duration: ${transcriptData.durationMinutes} minutes`,\n `Summary: ${notes.summary}`,\n `Key Concepts: ${(notes.key_concepts || []).join(', ')}`,\n '',\n 'Full Transcript:',\n transcriptData.fullText\n].join('\\n');\n\nreturn [{\n json: {\n className: transcriptData.className,\n classSlug: transcriptData.classSlug,\n lectureTitle: transcriptData.lectureTitle,\n durationMinutes: transcriptData.durationMinutes,\n notes,\n ragText,\n source: `class-${transcriptData.classSlug}-${Date.now()}`,\n metadata: {\n class_name: transcriptData.className,\n class_slug: transcriptData.classSlug,\n lecture_title: transcriptData.lectureTitle,\n duration_minutes: transcriptData.durationMinutes,\n type: 'class_recording',\n recorded_at: new Date().toISOString()\n }\n }\n}];"
|
||||
},
|
||||
"id": "71b0807d-03ba-45b2-b75b-b6fbeee5687e",
|
||||
"name": "Prepare for RAG",
|
||||
"type": "n8n-nodes-base.code",
|
||||
"typeVersion": 2,
|
||||
"position": [
|
||||
1104,
|
||||
0
|
||||
]
|
||||
},
|
||||
{
|
||||
"parameters": {
|
||||
"jsCode": "const input = $input.first().json;\nconst text = input.ragText || '';\nconst source = input.source;\nconst metadata = input.metadata;\n\nconst CHUNK_SIZE = 500;\nconst OVERLAP = 50;\n\nconst chunks = [];\nconst sentences = text.split(/(?<=[.!?])\\s+/);\nlet current = '';\n\nfor (const sentence of sentences) {\n if ((current + ' ' + sentence).length > CHUNK_SIZE && current.length > 0) {\n chunks.push(current.trim());\n const words = current.split(' ');\n current = words.slice(-OVERLAP).join(' ') + ' ' + sentence;\n } else {\n current = current ? current + ' ' + sentence : sentence;\n }\n}\nif (current.trim()) chunks.push(current.trim());\n\nreturn chunks.map((chunk, i) => ({\n json: {\n chunk,\n chunkIndex: i,\n totalChunks: chunks.length,\n source,\n metadata\n }\n}));"
|
||||
},
|
||||
"id": "9e4bbe70-72b0-4955-848f-0d1b62ae94a4",
|
||||
"name": "Chunk Text",
|
||||
"type": "n8n-nodes-base.code",
|
||||
"typeVersion": 2,
|
||||
"position": [
|
||||
1328,
|
||||
0
|
||||
]
|
||||
},
|
||||
{
|
||||
"parameters": {
|
||||
"method": "POST",
|
||||
"url": "http://10.5.1.6:11434/v1/embeddings",
|
||||
"sendHeaders": true,
|
||||
"headerParameters": {
|
||||
"parameters": [
|
||||
{
|
||||
"name": "Content-Type",
|
||||
"value": "application/json"
|
||||
}
|
||||
]
|
||||
},
|
||||
"sendBody": true,
|
||||
"specifyBody": "json",
|
||||
"jsonBody": "={\n \"model\": \"nomic-embed-text\",\n \"input\": \"{{ $json.chunk }}\"\n}",
|
||||
"options": {
|
||||
"batching": {
|
||||
"batch": {
|
||||
"batchSize": 5,
|
||||
"batchInterval": 500
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"id": "4ae1ab11-c76f-4478-84e0-7bcdfb20b83b",
|
||||
"name": "Get Embedding",
|
||||
"type": "n8n-nodes-base.httpRequest",
|
||||
"typeVersion": 4.2,
|
||||
"position": [
|
||||
1552,
|
||||
0
|
||||
]
|
||||
},
|
||||
{
|
||||
"parameters": {
|
||||
"jsCode": "const embeddingResponse = $input.first().json;\nconst chunkData = $('Chunk Text').item;\nconst embedding = embeddingResponse.data?.[0]?.embedding || [];\n\nconst pointId = 'xxxxxxxx-xxxx-4xxx-yxxx-xxxxxxxxxxxx'.replace(/[xy]/g, c => {\n const r = Math.random() * 16 | 0;\n return (c === 'x' ? r : (r & 0x3 | 0x8)).toString(16);\n});\n\nreturn [{\n json: {\n id: pointId,\n vector: embedding,\n payload: {\n text: chunkData.json.chunk,\n source: chunkData.json.source,\n chunkIndex: chunkData.json.chunkIndex,\n totalChunks: chunkData.json.totalChunks,\n metadata: chunkData.json.metadata,\n ingested_at: new Date().toISOString()\n }\n }\n}];"
|
||||
},
|
||||
"id": "d4232b02-23fc-4cbd-ac2d-2fdbfbe9583a",
|
||||
"name": "Prepare Qdrant Point",
|
||||
"type": "n8n-nodes-base.code",
|
||||
"typeVersion": 2,
|
||||
"position": [
|
||||
1760,
|
||||
0
|
||||
]
|
||||
},
|
||||
{
|
||||
"parameters": {
|
||||
"method": "PUT",
|
||||
"url": "http://10.5.1.6:6333/collections/knowledge_base/points",
|
||||
"sendHeaders": true,
|
||||
"headerParameters": {
|
||||
"parameters": [
|
||||
{
|
||||
"name": "Content-Type",
|
||||
"value": "application/json"
|
||||
}
|
||||
]
|
||||
},
|
||||
"sendBody": true,
|
||||
"specifyBody": "json",
|
||||
"jsonBody": "={\n \"points\": [\n {\n \"id\": \"{{ $json.id }}\",\n \"vector\": {{ JSON.stringify($json.vector) }},\n \"payload\": {{ JSON.stringify($json.payload) }}\n }\n ]\n}",
|
||||
"options": {}
|
||||
},
|
||||
"id": "845a2aa0-2546-431d-96e8-b36875d1fe7d",
|
||||
"name": "Upsert to Qdrant",
|
||||
"type": "n8n-nodes-base.httpRequest",
|
||||
"typeVersion": 4.2,
|
||||
"position": [
|
||||
1984,
|
||||
0
|
||||
]
|
||||
},
|
||||
{
|
||||
"parameters": {
|
||||
"jsCode": "const rag = $('Prepare for RAG').first().json;\nreturn [{\n json: {\n title: `\ud83c\udf93 ${rag.className} Lecture Processed`,\n message: `${rag.lectureTitle} (${rag.durationMinutes}min)\\nSummary: ${rag.notes.summary}\\nKey concepts: ${(rag.notes.key_concepts || []).slice(0, 5).join(', ')}`,\n priority: 5\n }\n}];"
|
||||
},
|
||||
"id": "fmt08-0001-0000-0000-000000000008",
|
||||
"name": "Format Gotify Message",
|
||||
"type": "n8n-nodes-base.code",
|
||||
"typeVersion": 2,
|
||||
"position": [
|
||||
1968,
|
||||
0
|
||||
]
|
||||
},
|
||||
{
|
||||
"parameters": {
|
||||
"message": "={{ $json.message }}",
|
||||
"additionalFields": {
|
||||
"priority": "={{ $json.priority }}",
|
||||
"title": "={{ $json.title }}"
|
||||
},
|
||||
"options": {
|
||||
"contentType": "text/plain"
|
||||
}
|
||||
},
|
||||
"id": "5b31b660-c414-40d1-82a5-d68511203133",
|
||||
"name": "Notify via Gotify",
|
||||
"type": "n8n-nodes-base.gotify",
|
||||
"typeVersion": 1,
|
||||
"position": [
|
||||
2208,
|
||||
0
|
||||
],
|
||||
"credentials": {
|
||||
"gotifyApi": {
|
||||
"id": "SETUP_REQUIRED",
|
||||
"name": "Class Recordings"
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"parameters": {
|
||||
"jsCode": "const rag = $('Prepare for RAG').first().json;\nconst notes = rag.notes || {};\nconst concepts = (notes.key_concepts || []).map((c,i) => `${i+1}. ${c}`).join('\\n');\nconst defs = Object.entries(notes.definitions || {}).map(([k,v]) => `${k}: ${v}`).join('\\n');\nconst assignments = (notes.assignments || []).map((a,i) => `${i+1}. ${a}`).join('\\n');\nconst content = [\n `Lecture Notes: ${rag.lectureTitle}`,\n `Class: ${rag.className}`,\n `Duration: ${rag.durationMinutes} minutes`,\n `Date: ${new Date().toISOString()}`, '',\n `Summary`, `=======`, notes.summary || '', '',\n `Key Concepts`, `=============`, concepts,\n ...(defs ? ['', `Definitions`, `===========`, defs] : []),\n ...(assignments ? ['', `Assignments`, `===========`, assignments] : []),\n].join('\\n');\nconst title = `${rag.className} \u2014 ${rag.lectureTitle}`;\nreturn [{ binary: { document: { data: Buffer.from(content,'utf-8').toString('base64'), mimeType:'text/plain', fileName:`${title}.txt` }}, json: { title, className: rag.className } }];\n"
|
||||
},
|
||||
"id": "95aea3a7-3b15-4196-9e4a-dc8f6ed8d49e",
|
||||
"name": "Prepare Notes Doc",
|
||||
"type": "n8n-nodes-base.code",
|
||||
"typeVersion": 2,
|
||||
"position": [
|
||||
2208,
|
||||
-200
|
||||
]
|
||||
},
|
||||
{
|
||||
"parameters": {
|
||||
"method": "POST",
|
||||
"url": "https://paperless.paccoco.com/api/documents/post_document/",
|
||||
"options": {},
|
||||
"sendHeaders": true,
|
||||
"headerParameters": {
|
||||
"parameters": [
|
||||
{
|
||||
"name": "Authorization",
|
||||
"value": "Bearer ${PAPERLESS_API_TOKEN}"
|
||||
}
|
||||
]
|
||||
},
|
||||
"sendBody": true,
|
||||
"contentType": "multipart-form-data",
|
||||
"bodyParameters": {
|
||||
"parameters": [
|
||||
{
|
||||
"name": "document",
|
||||
"value": "data",
|
||||
"parameterType": "formBinaryData"
|
||||
},
|
||||
{
|
||||
"name": "title",
|
||||
"value": "={{ $json.title }}"
|
||||
},
|
||||
{
|
||||
"name": "tags[]",
|
||||
"value": "${PAPERLESS_TAG_LECTURE_NOTES}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
"id": "c902a153-ab26-4518-af2a-585f0b72fe6c",
|
||||
"name": "Save Notes to Paperless",
|
||||
"type": "n8n-nodes-base.httpRequest",
|
||||
"typeVersion": 4.2,
|
||||
"position": [
|
||||
2432,
|
||||
-200
|
||||
],
|
||||
"continueOnFail": true
|
||||
}
|
||||
],
|
||||
"connections": {
|
||||
"Class Upload Webhook": {
|
||||
"main": [
|
||||
[
|
||||
{
|
||||
"node": "Extract Class Info",
|
||||
"type": "main",
|
||||
"index": 0
|
||||
}
|
||||
]
|
||||
]
|
||||
},
|
||||
"Extract Class Info": {
|
||||
"main": [
|
||||
[
|
||||
{
|
||||
"node": "Whisper Transcribe",
|
||||
"type": "main",
|
||||
"index": 0
|
||||
}
|
||||
]
|
||||
]
|
||||
},
|
||||
"Whisper Transcribe": {
|
||||
"main": [
|
||||
[
|
||||
{
|
||||
"node": "Format Transcript",
|
||||
"type": "main",
|
||||
"index": 0
|
||||
}
|
||||
]
|
||||
]
|
||||
},
|
||||
"Format Transcript": {
|
||||
"main": [
|
||||
[
|
||||
{
|
||||
"node": "AI Extract Notes",
|
||||
"type": "main",
|
||||
"index": 0
|
||||
}
|
||||
]
|
||||
]
|
||||
},
|
||||
"AI Extract Notes": {
|
||||
"main": [
|
||||
[
|
||||
{
|
||||
"node": "Prepare for RAG",
|
||||
"type": "main",
|
||||
"index": 0
|
||||
}
|
||||
]
|
||||
]
|
||||
},
|
||||
"Prepare for RAG": {
|
||||
"main": [
|
||||
[
|
||||
{
|
||||
"node": "Chunk Text",
|
||||
"type": "main",
|
||||
"index": 0
|
||||
}
|
||||
]
|
||||
]
|
||||
},
|
||||
"Chunk Text": {
|
||||
"main": [
|
||||
[
|
||||
{
|
||||
"node": "Get Embedding",
|
||||
"type": "main",
|
||||
"index": 0
|
||||
}
|
||||
]
|
||||
]
|
||||
},
|
||||
"Get Embedding": {
|
||||
"main": [
|
||||
[
|
||||
{
|
||||
"node": "Prepare Qdrant Point",
|
||||
"type": "main",
|
||||
"index": 0
|
||||
}
|
||||
]
|
||||
]
|
||||
},
|
||||
"Prepare Qdrant Point": {
|
||||
"main": [
|
||||
[
|
||||
{
|
||||
"node": "Upsert to Qdrant",
|
||||
"type": "main",
|
||||
"index": 0
|
||||
}
|
||||
]
|
||||
]
|
||||
},
|
||||
"Upsert to Qdrant": {
|
||||
"main": [
|
||||
[
|
||||
{
|
||||
"node": "Format Gotify Message",
|
||||
"type": "main",
|
||||
"index": 0
|
||||
},
|
||||
{
|
||||
"node": "Prepare Notes Doc",
|
||||
"type": "main",
|
||||
"index": 0
|
||||
}
|
||||
]
|
||||
]
|
||||
},
|
||||
"Format Gotify Message": {
|
||||
"main": [
|
||||
[
|
||||
{
|
||||
"node": "Notify via Gotify",
|
||||
"type": "main",
|
||||
"index": 0
|
||||
}
|
||||
]
|
||||
]
|
||||
},
|
||||
"Prepare Notes Doc": {
|
||||
"main": [
|
||||
[
|
||||
{
|
||||
"node": "Save Notes to Paperless",
|
||||
"type": "main",
|
||||
"index": 0
|
||||
}
|
||||
]
|
||||
]
|
||||
}
|
||||
},
|
||||
"settings": {
|
||||
"executionOrder": "v1",
|
||||
"binaryMode": "separate"
|
||||
},
|
||||
"staticData": null,
|
||||
"meta": null,
|
||||
"pinData": {},
|
||||
"versionId": "d64293a7-5537-4b01-a11b-6b838d6cacbf",
|
||||
"activeVersionId": "d64293a7-5537-4b01-a11b-6b838d6cacbf",
|
||||
"versionCounter": 11,
|
||||
"triggerCount": 1,
|
||||
"tags": [
|
||||
{
|
||||
"updatedAt": "2026-05-07T00:49:56.081Z",
|
||||
"createdAt": "2026-05-07T00:49:56.081Z",
|
||||
"id": "Q9lCYtCOuy9XmeyL",
|
||||
"name": "school"
|
||||
},
|
||||
{
|
||||
"updatedAt": "2026-05-06T22:39:59.716Z",
|
||||
"createdAt": "2026-05-06T22:39:59.716Z",
|
||||
"id": "tuPxDoKFqpVrrQoX",
|
||||
"name": "whisper"
|
||||
},
|
||||
{
|
||||
"updatedAt": "2026-05-06T22:39:33.284Z",
|
||||
"createdAt": "2026-05-06T22:39:33.284Z",
|
||||
"id": "7uuEyQrIlcuruZBk",
|
||||
"name": "rag"
|
||||
},
|
||||
{
|
||||
"updatedAt": "2026-05-06T22:38:51.678Z",
|
||||
"createdAt": "2026-05-06T22:38:51.678Z",
|
||||
"id": "S9FxzVfHLsHmyzyt",
|
||||
"name": "ai"
|
||||
}
|
||||
],
|
||||
"shared": [
|
||||
{
|
||||
"updatedAt": "2026-05-07T01:16:50.340Z",
|
||||
"createdAt": "2026-05-07T01:16:50.340Z",
|
||||
"role": "workflow:owner",
|
||||
"workflowId": "91I278E72FAaQenD",
|
||||
"projectId": "dxCRnBdX5uJizCGa",
|
||||
"project": {
|
||||
"updatedAt": "2026-05-06T01:10:37.484Z",
|
||||
"createdAt": "2026-05-06T01:08:07.721Z",
|
||||
"id": "dxCRnBdX5uJizCGa",
|
||||
"name": "Wilfred Fizzlepoof <admin@paccoco.com>",
|
||||
"type": "personal",
|
||||
"icon": null,
|
||||
"description": null,
|
||||
"creatorId": "47b2227f-2fc2-4a08-bd35-ea157b92df0d"
|
||||
}
|
||||
}
|
||||
],
|
||||
"versionMetadata": {
|
||||
"name": "Version d64293a7",
|
||||
"description": ""
|
||||
}
|
||||
}
|
||||
292
n8n-workflows/14-paperless-inbox-reminder.json
Normal file
292
n8n-workflows/14-paperless-inbox-reminder.json
Normal file
@@ -0,0 +1,292 @@
|
||||
{
|
||||
"name": "Paperless Inbox Reminder",
|
||||
"active": true,
|
||||
"isArchived": false,
|
||||
"nodes": [
|
||||
{
|
||||
"parameters": {
|
||||
"rule": {
|
||||
"interval": [
|
||||
{
|
||||
"field": "weeks",
|
||||
"weeksInterval": 1,
|
||||
"triggerAtDay": [
|
||||
1
|
||||
],
|
||||
"triggerAtHour": 9,
|
||||
"triggerAtMinute": 0
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
"id": "41404371-29ed-4660-9989-a5cbd890555b",
|
||||
"name": "Every Monday 9 AM",
|
||||
"type": "n8n-nodes-base.scheduleTrigger",
|
||||
"typeVersion": 1.2,
|
||||
"position": [
|
||||
0,
|
||||
0
|
||||
]
|
||||
},
|
||||
{
|
||||
"parameters": {
|
||||
"method": "GET",
|
||||
"url": "https://paperless.paccoco.com/api/documents/",
|
||||
"sendHeaders": true,
|
||||
"headerParameters": {
|
||||
"parameters": [
|
||||
{
|
||||
"name": "Authorization",
|
||||
"value": "Bearer ={{ $env.PAPERLESS_API_TOKEN }}"
|
||||
}
|
||||
]
|
||||
},
|
||||
"sendQuery": true,
|
||||
"queryParameters": {
|
||||
"parameters": [
|
||||
{
|
||||
"name": "is_tagged",
|
||||
"value": "0"
|
||||
},
|
||||
{
|
||||
"name": "page_size",
|
||||
"value": "25"
|
||||
},
|
||||
{
|
||||
"name": "ordering",
|
||||
"value": "created"
|
||||
},
|
||||
{
|
||||
"name": "fields",
|
||||
"value": "id,title,created,document_type,tags"
|
||||
}
|
||||
]
|
||||
},
|
||||
"options": {}
|
||||
},
|
||||
"id": "fdcd5728-6d27-445f-961b-18f204297469",
|
||||
"name": "Fetch Inbox Docs",
|
||||
"type": "n8n-nodes-base.httpRequest",
|
||||
"typeVersion": 4.2,
|
||||
"position": [
|
||||
224,
|
||||
0
|
||||
]
|
||||
},
|
||||
{
|
||||
"parameters": {
|
||||
"jsCode": "const data = $input.first().json;\nconst docs = data.results || [];\nconst total = data.count || 0;\n\n// Oldest 5 for the reminder body\nconst oldest = docs.slice(0, 5).map(d => {\n const date = d.created ? d.created.substring(0, 10) : 'unknown date';\n return ` \u2022 ${d.title || 'Untitled'} (added ${date})`;\n});\n\nreturn [{\n json: {\n total,\n hasItems: total > 0,\n oldestList: oldest.join('\\n'),\n remaining: Math.max(0, total - 5)\n }\n}];"
|
||||
},
|
||||
"id": "8fbe6be0-1ed3-4126-8f21-c3f4efb161c6",
|
||||
"name": "Parse Response",
|
||||
"type": "n8n-nodes-base.code",
|
||||
"typeVersion": 2,
|
||||
"position": [
|
||||
448,
|
||||
0
|
||||
]
|
||||
},
|
||||
{
|
||||
"parameters": {
|
||||
"conditions": {
|
||||
"options": {
|
||||
"caseSensitive": true,
|
||||
"leftValue": "",
|
||||
"typeValidation": "strict"
|
||||
},
|
||||
"conditions": [
|
||||
{
|
||||
"id": "efdd4f81-1976-4d26-a565-d110bfeba1df",
|
||||
"leftValue": "={{ $json.hasItems }}",
|
||||
"rightValue": true,
|
||||
"operator": {
|
||||
"type": "boolean",
|
||||
"operation": "equal"
|
||||
}
|
||||
}
|
||||
],
|
||||
"combinator": "and"
|
||||
}
|
||||
},
|
||||
"id": "e24344a1-e857-462c-b96a-842169e83a6d",
|
||||
"name": "Has Inbox Items?",
|
||||
"type": "n8n-nodes-base.if",
|
||||
"typeVersion": 2.2,
|
||||
"position": [
|
||||
672,
|
||||
0
|
||||
]
|
||||
},
|
||||
{
|
||||
"parameters": {
|
||||
"jsCode": "const d = $input.first().json;\nconst extra = d.remaining > 0 ? `\\n \u2026 and ${d.remaining} more` : '';\nconst body = `${d.total} untagged document${d.total !== 1 ? 's' : ''} in Paperless inbox:\\n${d.oldestList}${extra}\\n\\nhttps://paperless.paccoco.com`;\nreturn [{ json: { title: '\ud83d\udce5 Paperless Inbox Needs Attention', message: body, priority: 5 } }];"
|
||||
},
|
||||
"id": "897af9c9-4429-445f-8160-a03a14a318a3",
|
||||
"name": "Format Reminder",
|
||||
"type": "n8n-nodes-base.code",
|
||||
"typeVersion": 2,
|
||||
"position": [
|
||||
896,
|
||||
-120
|
||||
]
|
||||
},
|
||||
{
|
||||
"parameters": {
|
||||
"jsCode": "return [{ json: { title: '\u2705 Paperless Inbox Clear', message: 'No untagged documents \u2014 inbox is clean.', priority: 1 } }];"
|
||||
},
|
||||
"id": "bd8fe88e-7876-45dd-8344-983674fba79d",
|
||||
"name": "Format Clear",
|
||||
"type": "n8n-nodes-base.code",
|
||||
"typeVersion": 2,
|
||||
"position": [
|
||||
896,
|
||||
120
|
||||
]
|
||||
},
|
||||
{
|
||||
"parameters": {
|
||||
"message": "={{ $json.message }}",
|
||||
"additionalFields": {
|
||||
"priority": "={{ $json.priority }}",
|
||||
"title": "={{ $json.title }}"
|
||||
},
|
||||
"options": {
|
||||
"contentType": "text/plain"
|
||||
}
|
||||
},
|
||||
"id": "74860e7f-42e3-448f-9e46-5cfb6f12cdc2",
|
||||
"name": "Push to Gotify (Reminder)",
|
||||
"type": "n8n-nodes-base.gotify",
|
||||
"typeVersion": 1,
|
||||
"position": [
|
||||
1120,
|
||||
-120
|
||||
],
|
||||
"credentials": {
|
||||
"gotifyApi": {
|
||||
"id": "SETUP_REQUIRED",
|
||||
"name": "Reminders"
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"parameters": {
|
||||
"message": "={{ $json.message }}",
|
||||
"additionalFields": {
|
||||
"priority": "={{ $json.priority }}",
|
||||
"title": "={{ $json.title }}"
|
||||
},
|
||||
"options": {
|
||||
"contentType": "text/plain"
|
||||
}
|
||||
},
|
||||
"id": "b1dbd39b-87c3-43ec-9cec-f63eabbd0fa5",
|
||||
"name": "Push to Gotify (Clear)",
|
||||
"type": "n8n-nodes-base.gotify",
|
||||
"typeVersion": 1,
|
||||
"position": [
|
||||
1120,
|
||||
120
|
||||
],
|
||||
"credentials": {
|
||||
"gotifyApi": {
|
||||
"id": "SETUP_REQUIRED",
|
||||
"name": "Reminders"
|
||||
}
|
||||
}
|
||||
}
|
||||
],
|
||||
"connections": {
|
||||
"Every Monday 9 AM": {
|
||||
"main": [
|
||||
[
|
||||
{
|
||||
"node": "Fetch Inbox Docs",
|
||||
"type": "main",
|
||||
"index": 0
|
||||
}
|
||||
]
|
||||
]
|
||||
},
|
||||
"Fetch Inbox Docs": {
|
||||
"main": [
|
||||
[
|
||||
{
|
||||
"node": "Parse Response",
|
||||
"type": "main",
|
||||
"index": 0
|
||||
}
|
||||
]
|
||||
]
|
||||
},
|
||||
"Parse Response": {
|
||||
"main": [
|
||||
[
|
||||
{
|
||||
"node": "Has Inbox Items?",
|
||||
"type": "main",
|
||||
"index": 0
|
||||
}
|
||||
]
|
||||
]
|
||||
},
|
||||
"Has Inbox Items?": {
|
||||
"main": [
|
||||
[
|
||||
{
|
||||
"node": "Format Reminder",
|
||||
"type": "main",
|
||||
"index": 0
|
||||
}
|
||||
],
|
||||
[
|
||||
{
|
||||
"node": "Format Clear",
|
||||
"type": "main",
|
||||
"index": 0
|
||||
}
|
||||
]
|
||||
]
|
||||
},
|
||||
"Format Reminder": {
|
||||
"main": [
|
||||
[
|
||||
{
|
||||
"node": "Push to Gotify (Reminder)",
|
||||
"type": "main",
|
||||
"index": 0
|
||||
}
|
||||
]
|
||||
]
|
||||
},
|
||||
"Format Clear": {
|
||||
"main": [
|
||||
[
|
||||
{
|
||||
"node": "Push to Gotify (Clear)",
|
||||
"type": "main",
|
||||
"index": 0
|
||||
}
|
||||
]
|
||||
]
|
||||
}
|
||||
},
|
||||
"settings": {
|
||||
"executionOrder": "v1",
|
||||
"binaryMode": "separate"
|
||||
},
|
||||
"staticData": null,
|
||||
"meta": null,
|
||||
"pinData": {},
|
||||
"tags": [
|
||||
{
|
||||
"name": "reminders"
|
||||
},
|
||||
{
|
||||
"name": "paperless"
|
||||
},
|
||||
{
|
||||
"name": "scheduled"
|
||||
}
|
||||
]
|
||||
}
|
||||
383
n8n-workflows/15-n8n-health-monitor.json
Normal file
383
n8n-workflows/15-n8n-health-monitor.json
Normal file
@@ -0,0 +1,383 @@
|
||||
{
|
||||
"name": "n8n Self-Health Monitor",
|
||||
"active": true,
|
||||
"isArchived": false,
|
||||
"nodes": [
|
||||
{
|
||||
"parameters": {
|
||||
"rule": {
|
||||
"interval": [
|
||||
{
|
||||
"field": "minutes",
|
||||
"minutesInterval": 15
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
"id": "dabcb0ae-d513-410e-b779-4570df5c4a35",
|
||||
"name": "Every 15 Minutes",
|
||||
"type": "n8n-nodes-base.scheduleTrigger",
|
||||
"typeVersion": 1.2,
|
||||
"position": [
|
||||
0,
|
||||
0
|
||||
]
|
||||
},
|
||||
{
|
||||
"parameters": {
|
||||
"method": "GET",
|
||||
"url": "http://10.5.1.6:5678/api/v1/executions",
|
||||
"sendHeaders": true,
|
||||
"headerParameters": {
|
||||
"parameters": [
|
||||
{
|
||||
"name": "X-N8N-API-KEY",
|
||||
"value": "={{ $env.N8N_API_KEY }}"
|
||||
}
|
||||
]
|
||||
},
|
||||
"sendQuery": true,
|
||||
"queryParameters": {
|
||||
"parameters": [
|
||||
{
|
||||
"name": "status",
|
||||
"value": "error"
|
||||
},
|
||||
{
|
||||
"name": "includeData",
|
||||
"value": "false"
|
||||
},
|
||||
{
|
||||
"name": "limit",
|
||||
"value": "50"
|
||||
}
|
||||
]
|
||||
},
|
||||
"options": {}
|
||||
},
|
||||
"id": "806d75e9-b95a-499a-b997-a7f7dc046775",
|
||||
"name": "Fetch Failed Executions",
|
||||
"type": "n8n-nodes-base.httpRequest",
|
||||
"typeVersion": 4.2,
|
||||
"position": [
|
||||
224,
|
||||
0
|
||||
]
|
||||
},
|
||||
{
|
||||
"parameters": {
|
||||
"jsCode": "const data = $input.first().json;\nconst executions = data.data || [];\nconst oneHourAgo = Date.now() - (60 * 60 * 1000);\n\n// Filter to failures in the past hour\nconst recent = executions.filter(e => {\n const ts = new Date(e.startedAt || e.stoppedAt || 0).getTime();\n return ts >= oneHourAgo;\n});\n\n// Group by workflow name + error message for dedup key\nconst failures = recent.map(e => ({\n executionId: e.id,\n workflowId: e.workflowId,\n workflowName: e.workflowData?.name || `Workflow ${e.workflowId}`,\n startedAt: e.startedAt,\n stoppedAt: e.stoppedAt,\n errorMessage: e.data?.resultData?.error?.message || 'Unknown error'\n}));\n\n// Deduplicate by workflowName (report once per workflow per check)\nconst seen = new Set();\nconst unique = failures.filter(f => {\n if (seen.has(f.workflowName)) return false;\n seen.add(f.workflowName);\n return true;\n});\n\nreturn [{ json: { hasFailures: unique.length > 0, count: unique.length, failures: unique } }];"
|
||||
},
|
||||
"id": "f5d47910-5047-4c4f-aa84-a6907aa38551",
|
||||
"name": "Filter Recent Failures",
|
||||
"type": "n8n-nodes-base.code",
|
||||
"typeVersion": 2,
|
||||
"position": [
|
||||
448,
|
||||
0
|
||||
]
|
||||
},
|
||||
{
|
||||
"parameters": {
|
||||
"conditions": {
|
||||
"options": {
|
||||
"caseSensitive": true,
|
||||
"leftValue": "",
|
||||
"typeValidation": "strict"
|
||||
},
|
||||
"conditions": [
|
||||
{
|
||||
"id": "a76d10a7-1313-467e-9648-471b296229a3",
|
||||
"leftValue": "={{ $json.hasFailures }}",
|
||||
"rightValue": true,
|
||||
"operator": {
|
||||
"type": "boolean",
|
||||
"operation": "equal"
|
||||
}
|
||||
}
|
||||
],
|
||||
"combinator": "and"
|
||||
}
|
||||
},
|
||||
"id": "7c298433-ab89-435e-8955-73da6b986f46",
|
||||
"name": "Has Failures?",
|
||||
"type": "n8n-nodes-base.if",
|
||||
"typeVersion": 2.2,
|
||||
"position": [
|
||||
672,
|
||||
0
|
||||
]
|
||||
},
|
||||
{
|
||||
"parameters": {
|
||||
"jsCode": "const d = $input.first().json;\n// Fingerprint = sorted workflow names joined \u2014 stable across retriggers\nconst fingerprint = (d.failures || [])\n .map(f => f.workflowName)\n .sort()\n .join('|') || 'unknown';\nreturn [{ json: { ...d, fingerprint } }];"
|
||||
},
|
||||
"id": "d3609d45-ba9f-41d3-a9bd-8c558a719f10",
|
||||
"name": "Build Fingerprint",
|
||||
"type": "n8n-nodes-base.code",
|
||||
"typeVersion": 2,
|
||||
"position": [
|
||||
896,
|
||||
-120
|
||||
]
|
||||
},
|
||||
{
|
||||
"parameters": {
|
||||
"operation": "executeQuery",
|
||||
"query": "SELECT fingerprint, last_seen FROM n8n_health_dedup WHERE fingerprint = '{{ $json.fingerprint }}' AND last_seen > NOW() - INTERVAL '1 hour' LIMIT 1;",
|
||||
"options": {}
|
||||
},
|
||||
"id": "15a3dd71-bc5b-4396-9534-63518e097438",
|
||||
"name": "Lookup Dedup",
|
||||
"type": "n8n-nodes-base.postgres",
|
||||
"typeVersion": 2.5,
|
||||
"position": [
|
||||
1120,
|
||||
-120
|
||||
],
|
||||
"credentials": {
|
||||
"postgres": {
|
||||
"id": "n9svoXemqSZoNNUB",
|
||||
"name": "Postgres account"
|
||||
}
|
||||
},
|
||||
"continueOnFail": true
|
||||
},
|
||||
{
|
||||
"parameters": {
|
||||
"jsCode": "const alertData = $('Build Fingerprint').first().json;\nconst rows = $input.all().filter(r => r.json && r.json.fingerprint);\nconst alreadyNotified = rows.length > 0;\nreturn [{ json: { ...alertData, alreadyNotified, shouldNotify: !alreadyNotified } }];"
|
||||
},
|
||||
"id": "a92590e3-7044-4ba3-9b39-f6a9457c5fbe",
|
||||
"name": "Check Already Notified",
|
||||
"type": "n8n-nodes-base.code",
|
||||
"typeVersion": 2,
|
||||
"position": [
|
||||
1344,
|
||||
-120
|
||||
]
|
||||
},
|
||||
{
|
||||
"parameters": {
|
||||
"conditions": {
|
||||
"options": {
|
||||
"caseSensitive": true,
|
||||
"leftValue": "",
|
||||
"typeValidation": "strict"
|
||||
},
|
||||
"conditions": [
|
||||
{
|
||||
"id": "b1e77af5-6875-47dd-97af-8e13958a7217",
|
||||
"leftValue": "={{ $json.shouldNotify }}",
|
||||
"rightValue": true,
|
||||
"operator": {
|
||||
"type": "boolean",
|
||||
"operation": "equal"
|
||||
}
|
||||
}
|
||||
],
|
||||
"combinator": "and"
|
||||
}
|
||||
},
|
||||
"id": "39ac0eaf-73ce-4fe1-9c4b-d09ac920291d",
|
||||
"name": "Should Notify?",
|
||||
"type": "n8n-nodes-base.if",
|
||||
"typeVersion": 2.2,
|
||||
"position": [
|
||||
1568,
|
||||
-120
|
||||
]
|
||||
},
|
||||
{
|
||||
"parameters": {
|
||||
"operation": "executeQuery",
|
||||
"query": "INSERT INTO n8n_health_dedup (fingerprint, last_seen, workflow_names) VALUES ('{{ $json.fingerprint }}', NOW(), '{{ $json.failures.map(f => f.workflowName).join(\", \") }}') ON CONFLICT (fingerprint) DO UPDATE SET last_seen = NOW(), workflow_names = EXCLUDED.workflow_names;",
|
||||
"options": {}
|
||||
},
|
||||
"id": "0c3be155-0c3a-4354-b863-73a3f5bbf598",
|
||||
"name": "Upsert Dedup Record",
|
||||
"type": "n8n-nodes-base.postgres",
|
||||
"typeVersion": 2.5,
|
||||
"position": [
|
||||
1792,
|
||||
-120
|
||||
],
|
||||
"credentials": {
|
||||
"postgres": {
|
||||
"id": "n9svoXemqSZoNNUB",
|
||||
"name": "Postgres account"
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"parameters": {
|
||||
"jsCode": "const d = $('Build Fingerprint').first().json;\nconst lines = (d.failures || []).map(f =>\n ` \u2022 ${f.workflowName}\\n ${f.errorMessage.substring(0, 120)}`\n);\nconst body = `${d.count} workflow${d.count !== 1 ? 's' : ''} failed in the past hour:\\n\\n${lines.join('\\n')}\\n\\nhttps://n8n.paccoco.com`;\nreturn [{ json: { title: `\u26a0\ufe0f n8n: ${d.count} Workflow Failure${d.count !== 1 ? 's' : ''}`, message: body, priority: 7 } }];"
|
||||
},
|
||||
"id": "1c6b42bf-a38e-47b6-82ac-50c43ee9e583",
|
||||
"name": "Format Notification",
|
||||
"type": "n8n-nodes-base.code",
|
||||
"typeVersion": 2,
|
||||
"position": [
|
||||
2016,
|
||||
-120
|
||||
]
|
||||
},
|
||||
{
|
||||
"parameters": {
|
||||
"message": "={{ $json.message }}",
|
||||
"additionalFields": {
|
||||
"priority": "={{ $json.priority }}",
|
||||
"title": "={{ $json.title }}"
|
||||
},
|
||||
"options": {
|
||||
"contentType": "text/plain"
|
||||
}
|
||||
},
|
||||
"id": "09435c2f-9044-432a-8ca8-dd3ce3262c76",
|
||||
"name": "Push to Gotify",
|
||||
"type": "n8n-nodes-base.gotify",
|
||||
"typeVersion": 1,
|
||||
"position": [
|
||||
2240,
|
||||
-120
|
||||
],
|
||||
"credentials": {
|
||||
"gotifyApi": {
|
||||
"id": "SETUP_REQUIRED",
|
||||
"name": "n8n Health"
|
||||
}
|
||||
}
|
||||
}
|
||||
],
|
||||
"connections": {
|
||||
"Every 15 Minutes": {
|
||||
"main": [
|
||||
[
|
||||
{
|
||||
"node": "Fetch Failed Executions",
|
||||
"type": "main",
|
||||
"index": 0
|
||||
}
|
||||
]
|
||||
]
|
||||
},
|
||||
"Fetch Failed Executions": {
|
||||
"main": [
|
||||
[
|
||||
{
|
||||
"node": "Filter Recent Failures",
|
||||
"type": "main",
|
||||
"index": 0
|
||||
}
|
||||
]
|
||||
]
|
||||
},
|
||||
"Filter Recent Failures": {
|
||||
"main": [
|
||||
[
|
||||
{
|
||||
"node": "Has Failures?",
|
||||
"type": "main",
|
||||
"index": 0
|
||||
}
|
||||
]
|
||||
]
|
||||
},
|
||||
"Has Failures?": {
|
||||
"main": [
|
||||
[
|
||||
{
|
||||
"node": "Build Fingerprint",
|
||||
"type": "main",
|
||||
"index": 0
|
||||
}
|
||||
],
|
||||
[]
|
||||
]
|
||||
},
|
||||
"Build Fingerprint": {
|
||||
"main": [
|
||||
[
|
||||
{
|
||||
"node": "Lookup Dedup",
|
||||
"type": "main",
|
||||
"index": 0
|
||||
}
|
||||
]
|
||||
]
|
||||
},
|
||||
"Lookup Dedup": {
|
||||
"main": [
|
||||
[
|
||||
{
|
||||
"node": "Check Already Notified",
|
||||
"type": "main",
|
||||
"index": 0
|
||||
}
|
||||
]
|
||||
]
|
||||
},
|
||||
"Check Already Notified": {
|
||||
"main": [
|
||||
[
|
||||
{
|
||||
"node": "Should Notify?",
|
||||
"type": "main",
|
||||
"index": 0
|
||||
}
|
||||
]
|
||||
]
|
||||
},
|
||||
"Should Notify?": {
|
||||
"main": [
|
||||
[
|
||||
{
|
||||
"node": "Upsert Dedup Record",
|
||||
"type": "main",
|
||||
"index": 0
|
||||
}
|
||||
],
|
||||
[]
|
||||
]
|
||||
},
|
||||
"Upsert Dedup Record": {
|
||||
"main": [
|
||||
[
|
||||
{
|
||||
"node": "Format Notification",
|
||||
"type": "main",
|
||||
"index": 0
|
||||
}
|
||||
]
|
||||
]
|
||||
},
|
||||
"Format Notification": {
|
||||
"main": [
|
||||
[
|
||||
{
|
||||
"node": "Push to Gotify",
|
||||
"type": "main",
|
||||
"index": 0
|
||||
}
|
||||
]
|
||||
]
|
||||
}
|
||||
},
|
||||
"settings": {
|
||||
"executionOrder": "v1",
|
||||
"binaryMode": "separate"
|
||||
},
|
||||
"staticData": null,
|
||||
"meta": null,
|
||||
"pinData": {},
|
||||
"tags": [
|
||||
{
|
||||
"name": "monitoring"
|
||||
},
|
||||
{
|
||||
"name": "scheduled"
|
||||
},
|
||||
{
|
||||
"name": "n8n"
|
||||
}
|
||||
]
|
||||
}
|
||||
449
n8n-workflows/16-nfs-watchdog.json
Normal file
449
n8n-workflows/16-nfs-watchdog.json
Normal file
@@ -0,0 +1,449 @@
|
||||
{
|
||||
"name": "NFS Mount Watchdog + Auto-Heal",
|
||||
"active": true,
|
||||
"isArchived": false,
|
||||
"nodes": [
|
||||
{
|
||||
"parameters": {
|
||||
"rule": {
|
||||
"interval": [
|
||||
{
|
||||
"field": "minutes",
|
||||
"minutesInterval": 5
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
"id": "8c466a67-a0fa-42dc-8ed8-0918834fc3a7",
|
||||
"name": "Every 5 Minutes",
|
||||
"type": "n8n-nodes-base.scheduleTrigger",
|
||||
"typeVersion": 1.2,
|
||||
"position": [
|
||||
0,
|
||||
0
|
||||
]
|
||||
},
|
||||
{
|
||||
"parameters": {
|
||||
"command": "mountpoint -q /mnt/unraid/data/media && echo MEDIA_OK || echo MEDIA_MISSING; mountpoint -q /mnt/unraid/data/photos && echo PHOTOS_OK || echo PHOTOS_MISSING"
|
||||
},
|
||||
"id": "f5b2fee9-d67a-4da7-80dd-f72e6078aed9",
|
||||
"name": "Probe Mounts",
|
||||
"type": "n8n-nodes-base.ssh",
|
||||
"typeVersion": 1,
|
||||
"position": [
|
||||
224,
|
||||
0
|
||||
],
|
||||
"credentials": {
|
||||
"sshPrivateKey": {
|
||||
"id": "SETUP_REQUIRED",
|
||||
"name": "PD SSH"
|
||||
}
|
||||
},
|
||||
"continueOnFail": true
|
||||
},
|
||||
{
|
||||
"parameters": {
|
||||
"jsCode": "\nconst out = ($input.first().json.stdout || '') + ($input.first().json.stderr || '');\nconst mediaOk = out.includes('MEDIA_OK');\nconst photosOk = out.includes('PHOTOS_OK');\nconst missing = [];\nif (!mediaOk) missing.push({ mount: '/mnt/unraid/data/media', label: 'media',\n containers: ['plex','audiobookshelf'] });\nif (!photosOk) missing.push({ mount: '/mnt/unraid/data/photos', label: 'photos',\n containers: ['immich-server','immich-microservices','immich-machine-learning'] });\nreturn [{ json: { mediaOk, photosOk, anyMissing: missing.length > 0, missing } }];\n"
|
||||
},
|
||||
"id": "25bb3239-49cb-4570-b706-4022b9aa751c",
|
||||
"name": "Parse Mount Status",
|
||||
"type": "n8n-nodes-base.code",
|
||||
"typeVersion": 2,
|
||||
"position": [
|
||||
448,
|
||||
0
|
||||
]
|
||||
},
|
||||
{
|
||||
"parameters": {
|
||||
"conditions": {
|
||||
"options": {
|
||||
"caseSensitive": true,
|
||||
"leftValue": "",
|
||||
"typeValidation": "strict"
|
||||
},
|
||||
"conditions": [
|
||||
{
|
||||
"id": "442964fa-a29c-4aa2-98d1-9e25e927c496",
|
||||
"leftValue": "={{ $json.anyMissing }}",
|
||||
"rightValue": true,
|
||||
"operator": {
|
||||
"type": "boolean",
|
||||
"operation": "equal"
|
||||
}
|
||||
}
|
||||
],
|
||||
"combinator": "and"
|
||||
}
|
||||
},
|
||||
"id": "68b74832-85a7-4193-9382-a0c22a666e66",
|
||||
"name": "Any Missing?",
|
||||
"type": "n8n-nodes-base.if",
|
||||
"typeVersion": 2.2,
|
||||
"position": [
|
||||
672,
|
||||
0
|
||||
]
|
||||
},
|
||||
{
|
||||
"parameters": {
|
||||
"command": "sudo bash /mnt/tank/docker/scripts/mount-unraid-nfs.sh 2>&1"
|
||||
},
|
||||
"id": "b18c7537-0c2f-4122-bfe5-6469d6be6853",
|
||||
"name": "Run Mount Script",
|
||||
"type": "n8n-nodes-base.ssh",
|
||||
"typeVersion": 1,
|
||||
"position": [
|
||||
896,
|
||||
-120
|
||||
],
|
||||
"credentials": {
|
||||
"sshPrivateKey": {
|
||||
"id": "SETUP_REQUIRED",
|
||||
"name": "PD SSH"
|
||||
}
|
||||
},
|
||||
"continueOnFail": true
|
||||
},
|
||||
{
|
||||
"parameters": {
|
||||
"resume": "timeInterval",
|
||||
"unit": "seconds",
|
||||
"value": 10
|
||||
},
|
||||
"id": "2e927f83-9ddc-4816-8ee6-76d21ce21a5e",
|
||||
"name": "Wait 10s",
|
||||
"type": "n8n-nodes-base.wait",
|
||||
"typeVersion": 1.1,
|
||||
"position": [
|
||||
1120,
|
||||
-120
|
||||
],
|
||||
"webhookId": "4e89e8d0-7f08-4378-a2b7-63756665b039"
|
||||
},
|
||||
{
|
||||
"parameters": {
|
||||
"command": "mountpoint -q /mnt/unraid/data/media && echo MEDIA_OK || echo MEDIA_MISSING; mountpoint -q /mnt/unraid/data/photos && echo PHOTOS_OK || echo PHOTOS_MISSING"
|
||||
},
|
||||
"id": "d15c0c90-e94f-404a-a787-75f1b1b5befa",
|
||||
"name": "Re-probe Mounts",
|
||||
"type": "n8n-nodes-base.ssh",
|
||||
"typeVersion": 1,
|
||||
"position": [
|
||||
1344,
|
||||
-120
|
||||
],
|
||||
"credentials": {
|
||||
"sshPrivateKey": {
|
||||
"id": "SETUP_REQUIRED",
|
||||
"name": "PD SSH"
|
||||
}
|
||||
},
|
||||
"continueOnFail": true
|
||||
},
|
||||
{
|
||||
"parameters": {
|
||||
"jsCode": "\nconst out = ($input.first().json.stdout || '') + ($input.first().json.stderr || '');\nconst mediaOk = out.includes('MEDIA_OK');\nconst photosOk = out.includes('PHOTOS_OK');\n\n// What was originally missing (from step 3)\nconst originalMissing = $('Parse Mount Status').first().json.missing || [];\n\n// Which of the originally-missing mounts are now up?\nconst healed = originalMissing.filter(m =>\n (m.label === 'media' && mediaOk) ||\n (m.label === 'photos' && photosOk)\n);\nconst stillDown = originalMissing.filter(m =>\n (m.label === 'media' && !mediaOk) ||\n (m.label === 'photos' && !photosOk)\n);\n\n// Build flat container list to restart (only for healed mounts)\nconst containersToRestart = [...new Set(healed.flatMap(m => m.containers))];\nconst restartCmd = containersToRestart.length > 0\n ? `sudo docker restart ${containersToRestart.join(' ')}`\n : 'echo NO_RESTART_NEEDED';\n\nreturn [{ json: {\n allHealed: stillDown.length === 0,\n healed, stillDown,\n containersToRestart,\n restartCmd,\n originalMissing\n}}];\n"
|
||||
},
|
||||
"id": "48de38c2-df08-406f-ad61-809ca6e306a1",
|
||||
"name": "Parse Heal Result",
|
||||
"type": "n8n-nodes-base.code",
|
||||
"typeVersion": 2,
|
||||
"position": [
|
||||
1568,
|
||||
-120
|
||||
]
|
||||
},
|
||||
{
|
||||
"parameters": {
|
||||
"conditions": {
|
||||
"options": {
|
||||
"caseSensitive": true,
|
||||
"leftValue": "",
|
||||
"typeValidation": "strict"
|
||||
},
|
||||
"conditions": [
|
||||
{
|
||||
"id": "566dda6e-2234-406a-ad87-c886c7c5ae08",
|
||||
"leftValue": "={{ $json.allHealed }}",
|
||||
"rightValue": true,
|
||||
"operator": {
|
||||
"type": "boolean",
|
||||
"operation": "equal"
|
||||
}
|
||||
}
|
||||
],
|
||||
"combinator": "and"
|
||||
}
|
||||
},
|
||||
"id": "29a1b544-97b3-433f-9821-2bf5a8af95d2",
|
||||
"name": "Healed?",
|
||||
"type": "n8n-nodes-base.if",
|
||||
"typeVersion": 2.2,
|
||||
"position": [
|
||||
1792,
|
||||
-120
|
||||
]
|
||||
},
|
||||
{
|
||||
"parameters": {
|
||||
"command": "={{ $json.restartCmd }}"
|
||||
},
|
||||
"id": "8d791509-9e18-44c1-bf58-ed3027927710",
|
||||
"name": "Restart Containers",
|
||||
"type": "n8n-nodes-base.ssh",
|
||||
"typeVersion": 1,
|
||||
"position": [
|
||||
2016,
|
||||
-240
|
||||
],
|
||||
"credentials": {
|
||||
"sshPrivateKey": {
|
||||
"id": "SETUP_REQUIRED",
|
||||
"name": "PD SSH"
|
||||
}
|
||||
},
|
||||
"continueOnFail": true
|
||||
},
|
||||
{
|
||||
"parameters": {
|
||||
"jsCode": "\nconst d = $('Parse Heal Result').first().json;\nconst mountList = d.healed.map(m => ` \u2022 ${m.mount}`).join('\\n');\nconst containers = d.containersToRestart.join(', ') || 'none';\nconst body = `NFS mounts were missing and have been automatically remounted.\\n\\nRemounted:\\n${mountList}\\n\\nContainers restarted: ${containers}`;\nreturn [{ json: { title: '\ud83d\udd17 NFS Auto-Healed', message: body, priority: 5 } }];\n"
|
||||
},
|
||||
"id": "75c8976f-d622-43cd-a332-72c32f0c7c60",
|
||||
"name": "Format Healed Notify",
|
||||
"type": "n8n-nodes-base.code",
|
||||
"typeVersion": 2,
|
||||
"position": [
|
||||
2240,
|
||||
-240
|
||||
]
|
||||
},
|
||||
{
|
||||
"parameters": {
|
||||
"message": "={{ $json.message }}",
|
||||
"additionalFields": {
|
||||
"priority": "={{ $json.priority }}",
|
||||
"title": "={{ $json.title }}"
|
||||
},
|
||||
"options": {
|
||||
"contentType": "text/plain"
|
||||
}
|
||||
},
|
||||
"id": "38705cd2-775b-4ef9-b470-4a7709f132e5",
|
||||
"name": "Gotify (Healed)",
|
||||
"type": "n8n-nodes-base.gotify",
|
||||
"typeVersion": 1,
|
||||
"position": [
|
||||
2464,
|
||||
-240
|
||||
],
|
||||
"credentials": {
|
||||
"gotifyApi": {
|
||||
"id": "SETUP_REQUIRED",
|
||||
"name": "Uptime / Infra"
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"parameters": {
|
||||
"jsCode": "\nconst d = $('Parse Heal Result').first().json;\nconst downList = d.stillDown.map(m => ` \u2022 ${m.mount}`).join('\\n');\nconst healedList = d.healed.length > 0\n ? `\\nPartially healed:\\n${d.healed.map(m => ' \u2022 ' + m.mount).join('\\n')}`\n : '';\nconst body = `NFS mount script ran but the following paths are still unavailable. Manual intervention required.\\n\\nStill down:\\n${downList}${healedList}\\n\\nCheck Serenity (TrueNAS) is up and NFS service is running.`;\nreturn [{ json: { title: '\ud83d\udea8 NFS Down \u2014 Manual Fix Needed', message: body, priority: 9 } }];\n"
|
||||
},
|
||||
"id": "074b984f-7fc2-45bc-b97f-f5db4d8d589b",
|
||||
"name": "Format Escalation",
|
||||
"type": "n8n-nodes-base.code",
|
||||
"typeVersion": 2,
|
||||
"position": [
|
||||
2016,
|
||||
-20
|
||||
]
|
||||
},
|
||||
{
|
||||
"parameters": {
|
||||
"message": "={{ $json.message }}",
|
||||
"additionalFields": {
|
||||
"priority": "={{ $json.priority }}",
|
||||
"title": "={{ $json.title }}"
|
||||
},
|
||||
"options": {
|
||||
"contentType": "text/plain"
|
||||
}
|
||||
},
|
||||
"id": "f929dd77-6072-4ac9-9dc4-11246110efed",
|
||||
"name": "Gotify (Escalation)",
|
||||
"type": "n8n-nodes-base.gotify",
|
||||
"typeVersion": 1,
|
||||
"position": [
|
||||
2240,
|
||||
-20
|
||||
],
|
||||
"credentials": {
|
||||
"gotifyApi": {
|
||||
"id": "SETUP_REQUIRED",
|
||||
"name": "Uptime / Infra"
|
||||
}
|
||||
}
|
||||
}
|
||||
],
|
||||
"connections": {
|
||||
"Every 5 Minutes": {
|
||||
"main": [
|
||||
[
|
||||
{
|
||||
"node": "Probe Mounts",
|
||||
"type": "main",
|
||||
"index": 0
|
||||
}
|
||||
]
|
||||
]
|
||||
},
|
||||
"Probe Mounts": {
|
||||
"main": [
|
||||
[
|
||||
{
|
||||
"node": "Parse Mount Status",
|
||||
"type": "main",
|
||||
"index": 0
|
||||
}
|
||||
]
|
||||
]
|
||||
},
|
||||
"Parse Mount Status": {
|
||||
"main": [
|
||||
[
|
||||
{
|
||||
"node": "Any Missing?",
|
||||
"type": "main",
|
||||
"index": 0
|
||||
}
|
||||
]
|
||||
]
|
||||
},
|
||||
"Any Missing?": {
|
||||
"main": [
|
||||
[
|
||||
{
|
||||
"node": "Run Mount Script",
|
||||
"type": "main",
|
||||
"index": 0
|
||||
}
|
||||
],
|
||||
[]
|
||||
]
|
||||
},
|
||||
"Run Mount Script": {
|
||||
"main": [
|
||||
[
|
||||
{
|
||||
"node": "Wait 10s",
|
||||
"type": "main",
|
||||
"index": 0
|
||||
}
|
||||
]
|
||||
]
|
||||
},
|
||||
"Wait 10s": {
|
||||
"main": [
|
||||
[
|
||||
{
|
||||
"node": "Re-probe Mounts",
|
||||
"type": "main",
|
||||
"index": 0
|
||||
}
|
||||
]
|
||||
]
|
||||
},
|
||||
"Re-probe Mounts": {
|
||||
"main": [
|
||||
[
|
||||
{
|
||||
"node": "Parse Heal Result",
|
||||
"type": "main",
|
||||
"index": 0
|
||||
}
|
||||
]
|
||||
]
|
||||
},
|
||||
"Parse Heal Result": {
|
||||
"main": [
|
||||
[
|
||||
{
|
||||
"node": "Healed?",
|
||||
"type": "main",
|
||||
"index": 0
|
||||
}
|
||||
]
|
||||
]
|
||||
},
|
||||
"Healed?": {
|
||||
"main": [
|
||||
[
|
||||
{
|
||||
"node": "Restart Containers",
|
||||
"type": "main",
|
||||
"index": 0
|
||||
}
|
||||
],
|
||||
[
|
||||
{
|
||||
"node": "Format Escalation",
|
||||
"type": "main",
|
||||
"index": 0
|
||||
}
|
||||
]
|
||||
]
|
||||
},
|
||||
"Restart Containers": {
|
||||
"main": [
|
||||
[
|
||||
{
|
||||
"node": "Format Healed Notify",
|
||||
"type": "main",
|
||||
"index": 0
|
||||
}
|
||||
]
|
||||
]
|
||||
},
|
||||
"Format Healed Notify": {
|
||||
"main": [
|
||||
[
|
||||
{
|
||||
"node": "Gotify (Healed)",
|
||||
"type": "main",
|
||||
"index": 0
|
||||
}
|
||||
]
|
||||
]
|
||||
},
|
||||
"Format Escalation": {
|
||||
"main": [
|
||||
[
|
||||
{
|
||||
"node": "Gotify (Escalation)",
|
||||
"type": "main",
|
||||
"index": 0
|
||||
}
|
||||
]
|
||||
]
|
||||
}
|
||||
},
|
||||
"settings": {
|
||||
"executionOrder": "v1",
|
||||
"binaryMode": "separate"
|
||||
},
|
||||
"staticData": null,
|
||||
"meta": null,
|
||||
"pinData": {},
|
||||
"tags": [
|
||||
{
|
||||
"name": "monitoring"
|
||||
},
|
||||
{
|
||||
"name": "infrastructure"
|
||||
},
|
||||
{
|
||||
"name": "scheduled"
|
||||
}
|
||||
]
|
||||
}
|
||||
242
n8n-workflows/README.md
Normal file
242
n8n-workflows/README.md
Normal file
@@ -0,0 +1,242 @@
|
||||
# n8n AI Workflows for Homelab
|
||||
|
||||
## Workflows Included
|
||||
|
||||
| # | Workflow | Trigger | What It Does |
|
||||
|---|---------|---------|--------------|
|
||||
| 01 | Grafana Alert → AI → Gotify | `POST /webhook/grafana-alert` | Receives Grafana alerts, AI-summarizes, deduplicates within 30 min, suppresses low-priority alerts overnight — pushes to Gotify |
|
||||
| 02 | RAG Pipeline | `POST /webhook/rag/ingest` + `POST /webhook/rag/query` | Ingest docs into Qdrant, query them with AI-powered answers |
|
||||
| 03 | Paperless AI Processing | `POST /webhook/paperless/new-document` | Auto-classifies & summarizes new Paperless docs, applies AI-suggested tags back to Paperless, notifies via Gotify |
|
||||
| 04 | Whisper Transcription | `POST /webhook/transcribe` + `POST /webhook/transcribe/summarize` | Transcribe audio → text, optionally with AI summary; notifies via Gotify when done and saves transcript to Paperless |
|
||||
| 05 | Paperless → RAG | `POST /webhook/paperless/rag-ingest` | Auto-ingests Paperless documents into Qdrant for RAG search; notifies via Gotify |
|
||||
| 06 | RSS Digest | Scheduled (every 6h) + `POST /webhook/scrape/summarize` | Fetches RSS feeds, deduplicates via Postgres, AI-summarizes per-feed, pushes to Gotify |
|
||||
| 07 | Git Commit Summarizer | `POST /webhook/git/push` | Receives git push webhooks, AI-summarizes changes, notifies via Gotify |
|
||||
| 08 | Class Recording → RAG | Watches `/mnt/data/class-recordings/` | Auto-transcribes new .wav files, extracts key notes, ingests into RAG by class, saves notes to Paperless, notifies via Gotify |
|
||||
| 14 | Paperless Inbox Reminder | Scheduled (Monday 9 AM) | Queries Paperless for untagged documents; sends Gotify reminder with oldest items listed, or a ✅ clear if inbox is empty |
|
||||
| 15 | n8n Self-Health Monitor | Scheduled (every 15 min) | Checks n8n execution history for failures in the past hour; deduplicates via Postgres; pushes Gotify alert with workflow name and error summary |
|
||||
| 16 | NFS Mount Watchdog | Scheduled (every 5 min) | Probes NFS mount points on PD; if missing: runs mount script, re-probes, restarts affected containers (Plex, Audiobookshelf, Immich); notifies via Gotify on heal or escalation |
|
||||
|
||||
## How to Import
|
||||
|
||||
1. Open n8n at https://n8n.paccoco.com
|
||||
2. Go to **Workflows** → click the **⋮** menu → **Import from File**
|
||||
3. Select each `.json` file from this folder
|
||||
4. Activate the workflow (toggle in top-right)
|
||||
|
||||
## Gotify Channel Map
|
||||
|
||||
Each workflow uses a dedicated Gotify app credential. Create these apps in Gotify and add matching credentials in n8n → Credentials:
|
||||
|
||||
| Gotify App | Used by |
|
||||
|------------|---------|
|
||||
| Gotify account | 01 (Grafana Alerts) |
|
||||
| Paperless | 03 |
|
||||
| Whisper | 04 |
|
||||
| Paperless RAG | 05 |
|
||||
| RSS Feed | 06 |
|
||||
| Git Commits | 07 |
|
||||
| Class Recordings | 08 |
|
||||
| Reminders | 14 |
|
||||
| n8n Health | 15 |
|
||||
| Uptime / Infra | 16 |
|
||||
|
||||
## One-Time Setup
|
||||
|
||||
### Postgres Dedup Tables (required for workflows 01, 15)
|
||||
|
||||
```sql
|
||||
-- Run via: sudo docker exec -it shared-postgres psql -U postgres -d n8n
|
||||
CREATE TABLE IF NOT EXISTS grafana_alert_dedup (
|
||||
fingerprint TEXT PRIMARY KEY,
|
||||
last_seen TIMESTAMPTZ DEFAULT NOW(),
|
||||
alert_name TEXT,
|
||||
host TEXT
|
||||
);
|
||||
CREATE TABLE IF NOT EXISTS n8n_health_dedup (
|
||||
fingerprint TEXT PRIMARY KEY,
|
||||
last_seen TIMESTAMPTZ DEFAULT NOW(),
|
||||
workflow_names TEXT
|
||||
);
|
||||
```
|
||||
|
||||
### n8n Environment Variables
|
||||
|
||||
Add these to n8n's `.env` and run `sudo docker compose up -d` to apply:
|
||||
|
||||
| Variable | Used by | How to get |
|
||||
|----------|---------|------------|
|
||||
| `PAPERLESS_API_TOKEN` | 04, 08 | Paperless → Settings → API token |
|
||||
| `PAPERLESS_TAG_TRANSCRIPTION` | 04 | Tag ID from Paperless API |
|
||||
| `PAPERLESS_TAG_LECTURE_NOTES` | 08 | Tag ID from Paperless API |
|
||||
| `N8N_API_KEY` | 15 | n8n → Settings → API → Create API Key |
|
||||
|
||||
### SSH Credential (required for workflow 16)
|
||||
|
||||
Workflow 16 SSHes into PD to probe mounts and restart containers. Set up key-based auth:
|
||||
|
||||
```bash
|
||||
# Run on PD — generate a dedicated key for n8n
|
||||
ssh-keygen -t ed25519 -f ~/.ssh/n8n_watchdog -N ""
|
||||
cat ~/.ssh/n8n_watchdog.pub >> ~/.ssh/authorized_keys
|
||||
cat ~/.ssh/n8n_watchdog # paste this into n8n → Credentials → SSH (Private Key)
|
||||
```
|
||||
|
||||
In n8n: Credentials → New → SSH (Private Key) → host `10.5.1.6`, user `truenas_admin`, paste private key. Name it `PD SSH`.
|
||||
|
||||
Workflow 16 also runs `sudo docker restart` and the mount script over SSH. Add a NOPASSWD sudoers rule so it doesn't hang:
|
||||
|
||||
```bash
|
||||
sudo visudo -f /etc/sudoers.d/n8n-watchdog
|
||||
# Add:
|
||||
# truenas_admin ALL=(ALL) NOPASSWD: /usr/bin/docker, /bin/bash /mnt/tank/docker/scripts/mount-unraid-nfs.sh
|
||||
```
|
||||
|
||||
### Qdrant Collection (required for workflows 02, 05, 08)
|
||||
|
||||
Create the `knowledge_base` collection before using the RAG workflows:
|
||||
|
||||
```bash
|
||||
curl -X PUT http://10.5.1.6:6333/collections/knowledge_base \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d '{
|
||||
"vectors": {
|
||||
"size": 768,
|
||||
"distance": "Cosine"
|
||||
}
|
||||
}'
|
||||
```
|
||||
|
||||
(768 dimensions matches `nomic-embed-text`. If you use a different embedding model, adjust the size.)
|
||||
|
||||
### Pull nomic-embed-text into Ollama (required for workflows 02, 05, 08)
|
||||
|
||||
```bash
|
||||
curl http://10.5.1.6:11434/api/pull -d '{"name": "nomic-embed-text"}'
|
||||
```
|
||||
|
||||
### Paperless Webhook (required for workflows 03 and 05)
|
||||
|
||||
In Paperless-NGX, set up post-consumption webhooks to POST to:
|
||||
```
|
||||
https://n8n.paccoco.com/webhook/paperless/new-document
|
||||
https://n8n.paccoco.com/webhook/paperless/rag-ingest
|
||||
```
|
||||
with body: `{"document_id": <id>}`
|
||||
|
||||
You can trigger both from the same Paperless event — workflow 03 classifies/tags the document, workflow 05 ingests it into RAG.
|
||||
|
||||
### Grafana Webhook (already configured)
|
||||
|
||||
Your Grafana contact point is already set to `https://n8n.paccoco.com/webhook/grafana-alert` — workflow 01 will work immediately once activated.
|
||||
|
||||
### Git Webhook (required for workflow 07)
|
||||
|
||||
In your Gitea/Forgejo/GitHub repo settings, add a webhook:
|
||||
- URL: `https://n8n.paccoco.com/webhook/git/push`
|
||||
- Content type: `application/json`
|
||||
- Events: Push only
|
||||
|
||||
Works with Gitea, Forgejo, GitHub, and GitLab webhook payloads.
|
||||
|
||||
## Usage Examples
|
||||
|
||||
### Ingest a document into RAG:
|
||||
```bash
|
||||
curl -X POST https://n8n.paccoco.com/webhook/rag/ingest \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d '{
|
||||
"text": "Your document content here...",
|
||||
"source": "my-notes",
|
||||
"metadata": {"topic": "homelab"}
|
||||
}'
|
||||
```
|
||||
|
||||
### Query your knowledge base:
|
||||
```bash
|
||||
curl -X POST https://n8n.paccoco.com/webhook/rag/query \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d '{"query": "How do I configure Qdrant?"}'
|
||||
```
|
||||
|
||||
### Transcribe audio:
|
||||
```bash
|
||||
curl -X POST https://n8n.paccoco.com/webhook/transcribe \
|
||||
-F "data=@recording.mp3"
|
||||
```
|
||||
|
||||
### Transcribe + AI summarize:
|
||||
```bash
|
||||
curl -X POST https://n8n.paccoco.com/webhook/transcribe/summarize \
|
||||
-F "data=@meeting.mp3"
|
||||
```
|
||||
|
||||
### Add a class recording (just drop a file):
|
||||
```
|
||||
/mnt/data/class-recordings/
|
||||
├── CS101-Java/
|
||||
│ ├── lecture-01-intro.wav
|
||||
│ ├── lecture-02-variables.wav
|
||||
│ └── lecture-03-loops.wav
|
||||
├── MATH201/
|
||||
│ └── lecture-01-derivatives.wav
|
||||
└── ENG102/
|
||||
└── essay-workshop.wav
|
||||
```
|
||||
|
||||
The workflow detects the class from the subfolder name and the lecture title from the filename.
|
||||
Just drop a `.wav` file into the right class folder and it auto-processes.
|
||||
|
||||
### Query RAG for class-specific content:
|
||||
```bash
|
||||
curl -X POST https://n8n.paccoco.com/webhook/rag/query \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d '{"query": "What did my CS101 lecture say about inheritance in Java?"}'
|
||||
```
|
||||
|
||||
The RAG system stores class name metadata with each chunk, so when you mention a class name in your query it will prioritize results from that class.
|
||||
|
||||
### Class Recordings Setup (Workflow 08)
|
||||
|
||||
Requires n8n volume mount to the NFS share. Add to n8n's docker-compose:
|
||||
```yaml
|
||||
volumes:
|
||||
- /mnt/tank/docker/appdata/n8n:/home/node/.n8n
|
||||
- /mnt/data/class-recordings:/data/class-recordings
|
||||
```
|
||||
|
||||
Create class subfolders:
|
||||
```bash
|
||||
mkdir -p /mnt/data/class-recordings/CS101-Java
|
||||
mkdir -p /mnt/data/class-recordings/MATH201
|
||||
```
|
||||
|
||||
Then restart n8n and activate the workflow. Any `.wav` dropped into a subfolder triggers automatic transcription + RAG ingest.
|
||||
|
||||
### Trigger RSS digest manually:
|
||||
```bash
|
||||
curl -X POST https://n8n.paccoco.com/webhook/scrape/summarize
|
||||
```
|
||||
|
||||
### Test git webhook:
|
||||
```bash
|
||||
curl -X POST https://n8n.paccoco.com/webhook/git/push \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d '{
|
||||
"ref": "refs/heads/main",
|
||||
"pusher": {"name": "john"},
|
||||
"repository": {"full_name": "john/my-project"},
|
||||
"commits": [
|
||||
{"id": "abc1234", "message": "Fix null pointer in UserService", "author": {"name": "john"}, "added": [], "modified": ["src/UserService.java"], "removed": []}
|
||||
]
|
||||
}'
|
||||
```
|
||||
|
||||
## Customizing RSS Feeds (Workflow 06)
|
||||
|
||||
Edit the "Define RSS Feeds" Code node to add/remove feeds. Default feeds:
|
||||
- Self-Hosted Show
|
||||
- r/selfhosted
|
||||
- r/homelab
|
||||
- Hacker News
|
||||
|
||||
Add any RSS feed URL you want monitored.
|
||||
121
n8n-workflows/class-watcher.sh
Normal file
121
n8n-workflows/class-watcher.sh
Normal file
@@ -0,0 +1,121 @@
|
||||
#!/bin/bash
|
||||
# class-watcher.sh — Watches /mnt/data/class-recordings for new .wav files
|
||||
# and POSTs them to the n8n class/upload webhook.
|
||||
# Saves the transcript + notes response as a .md file alongside the recording.
|
||||
#
|
||||
# Install:
|
||||
# 1. Copy to PD: scp class-watcher.sh truenas_admin@10.5.1.6:/tmp/class-watcher.sh
|
||||
# 2. sudo mv /tmp/class-watcher.sh /mnt/data/class-watcher.sh
|
||||
# 3. sudo chmod +x /mnt/data/class-watcher.sh
|
||||
# 4. sudo crontab -e
|
||||
# * * * * * /mnt/data/class-watcher.sh >> /var/log/class-watcher.log 2>&1
|
||||
#
|
||||
# Folder structure after processing:
|
||||
# /mnt/data/class-recordings/
|
||||
# └── CS101-Java/
|
||||
# ├── new-lecture.wav ← waiting to be processed
|
||||
# └── done/
|
||||
# ├── lecture-01-intro.wav ← already processed
|
||||
# └── lecture-01-intro.md ← transcript + notes
|
||||
|
||||
WATCH_DIR="/mnt/data/class-recordings"
|
||||
WEBHOOK_URL="http://10.5.1.6:5678/webhook/class/upload"
|
||||
|
||||
# Find all .wav files directly inside class subfolders (not in done/)
|
||||
find "$WATCH_DIR" -mindepth 2 -maxdepth 2 -name '*.wav' -type f | while read -r filepath; do
|
||||
# Skip files in 'done' directories
|
||||
if echo "$filepath" | grep -q '/done/'; then
|
||||
continue
|
||||
fi
|
||||
|
||||
# Extract class name from parent folder
|
||||
class_dir=$(dirname "$filepath")
|
||||
class_name=$(basename "$class_dir")
|
||||
file_name=$(basename "$filepath")
|
||||
lecture_title="${file_name%.wav}"
|
||||
|
||||
echo "[$(date)] Processing: $filepath (class: $class_name)"
|
||||
|
||||
# Create done folder
|
||||
mkdir -p "$class_dir/done"
|
||||
|
||||
# POST the file to n8n webhook and capture the response
|
||||
response=$(curl -s -X POST "$WEBHOOK_URL" \
|
||||
-H "X-Class-Name: $class_name" \
|
||||
-H "X-Lecture-Title: $lecture_title" \
|
||||
-F "data=@$filepath" \
|
||||
-F "class_name=$class_name" \
|
||||
-F "lecture_title=$lecture_title" \
|
||||
-F "filename=$file_name" \
|
||||
-w "\n%{http_code}" 2>/dev/null)
|
||||
|
||||
http_code=$(echo "$response" | tail -1)
|
||||
body=$(echo "$response" | sed '$d')
|
||||
|
||||
echo "[$(date)] HTTP $http_code"
|
||||
|
||||
# Save the transcript + notes as markdown
|
||||
md_file="$class_dir/done/${lecture_title}.md"
|
||||
|
||||
# Try to parse JSON response and format as markdown
|
||||
if command -v python3 &>/dev/null; then
|
||||
echo "$body" | python3 -c "
|
||||
import json, sys
|
||||
try:
|
||||
data = json.load(sys.stdin)
|
||||
# Handle array response from n8n
|
||||
if isinstance(data, list):
|
||||
data = data[0] if data else {}
|
||||
cn = data.get('className', '$class_name')
|
||||
lt = data.get('lectureTitle', '$lecture_title')
|
||||
dm = data.get('durationMinutes', '?')
|
||||
notes = data.get('notes', {})
|
||||
summary = notes.get('summary', 'No summary available')
|
||||
concepts = notes.get('key_concepts', [])
|
||||
defs = notes.get('definitions', [])
|
||||
assignments = notes.get('assignments', [])
|
||||
transcript = data.get('timestampedText', data.get('fullText', 'No transcript'))
|
||||
|
||||
print(f'# {cn} — {lt}')
|
||||
print(f'**Duration:** {dm} minutes')
|
||||
print()
|
||||
print('## Summary')
|
||||
print(summary)
|
||||
print()
|
||||
if concepts:
|
||||
print('## Key Concepts')
|
||||
for c in concepts:
|
||||
print(f'- {c}')
|
||||
print()
|
||||
if defs:
|
||||
print('## Definitions')
|
||||
for d in defs:
|
||||
if isinstance(d, dict):
|
||||
print(f'- **{d.get(\"term\", \"\")}**: {d.get(\"definition\", \"\")}')
|
||||
else:
|
||||
print(f'- {d}')
|
||||
print()
|
||||
if assignments:
|
||||
print('## Assignments')
|
||||
for a in assignments:
|
||||
print(f'- {a}')
|
||||
print()
|
||||
print('## Full Transcript')
|
||||
print(transcript)
|
||||
except Exception as e:
|
||||
print(f'# {\"$class_name\"} — {\"$lecture_title\"}')
|
||||
print()
|
||||
print('## Raw Response')
|
||||
print(sys.stdin.read() if not data else json.dumps(data, indent=2))
|
||||
" > "$md_file" 2>/dev/null
|
||||
else
|
||||
# Fallback: save raw JSON
|
||||
echo "$body" > "$md_file"
|
||||
fi
|
||||
|
||||
echo "[$(date)] Saved notes to: $md_file"
|
||||
|
||||
# Move wav to done folder
|
||||
mv "$filepath" "$class_dir/done/$file_name"
|
||||
echo "[$(date)] Moved to: $class_dir/done/$file_name"
|
||||
done
|
||||
Reference in New Issue
Block a user