Files

Fizzlepoof 5f56fe75d8 Update stale service IP references after VLAN 30 cutover

2026-05-22 20:36:12 +00:00

20 KiB

Raw Blame History

n8n AI Workflows for Homelab

Live Production Set

Live n8n was pruned on 2026-05-18 to remove superseded inactive copies plus the one-shot setup leftovers Temp Postgres Credential Probe and Temp School Intake Table Provision.

Active workflows after cleanup:

Grafana Alert → AI → Gotify v1.4
RAG Pipeline v1.0
Paperless AI Processing v1.0
Whisper Audio Transcription
Paperless → RAG v1.0
Git Commit → AI Summary → Gotify v1.0
Class Recording → Transcribe → RAG
Class Transcript Ingest v1.1
Paperless Inbox Reminder
n8n Self-Health Monitor
NFS Mount Watchdog + Auto-Heal v1.2
Paperless Intake Triage → Doris Review Queue
Telegram School Intake → Postgres → Paperless v1.2
Paperless School Intake Metadata Enrichment v1.1

Operational note: the live Paperless Inbox Reminder workflow needed its auth header corrected on 2026-05-18 so it uses Token $env.PAPERLESS_API_TOKEN instead of the broken bearer form that had drifted into the active copy.

Workflows Included

#	Workflow	Trigger	What It Does
01	Grafana Alert → AI → Gotify	`POST /webhook/grafana-alert`	Receives Grafana alerts, AI-summarizes, deduplicates within 30 min, suppresses low-priority alerts overnight — pushes to Gotify
02	RAG Pipeline	`POST /webhook/rag/ingest` + `POST /webhook/rag/query`	Ingest docs into Qdrant, query them with AI-powered answers
03	Paperless AI Processing	`POST /webhook/paperless/new-document`	Auto-classifies & summarizes new Paperless docs, applies AI-suggested tags back to Paperless, notifies via Gotify
04	Whisper Transcription	`POST /webhook/transcribe` + `POST /webhook/transcribe/summarize`	Transcribe audio → text, optionally with AI summary; notifies via Gotify when done and saves transcript to Paperless
05	Paperless → RAG	`POST /webhook/paperless/rag-ingest`	Auto-ingests Paperless documents into Qdrant for RAG search; notifies via Gotify
06	RSS Digest	Scheduled (every 6h) + `POST /webhook/scrape/summarize`	Fetches RSS feeds, deduplicates via Postgres, AI-summarizes per-feed, pushes to Gotify
07	Git Commit Summarizer	`POST /webhook/git/push`	Receives git push webhooks, AI-summarizes changes, notifies via Gotify
08	Class Recording → RAG	`POST /webhook/class/upload`	Transcribes uploaded .wav, extracts key notes, ingests into RAG by class, saves notes to Paperless, notifies via Gotify
14	Paperless Inbox Reminder	Scheduled (Monday 9 AM)	Queries Paperless for untagged documents; sends Gotify reminder with oldest items listed, or a ✅ clear if inbox is empty
15	n8n Self-Health Monitor	Scheduled (every 15 min)	Checks n8n execution history for failures in the past hour; deduplicates via Postgres; pushes Gotify alert with workflow name and error summary
16	NFS Mount Watchdog	Scheduled (every 5 min)	Probes NFS mount points on PD; if missing: runs mount script, re-probes, restarts affected containers (Plex, Audiobookshelf, Immich); notifies via Gotify on heal or escalation
17	Paperless Intake Triage → Doris Review Queue	`POST /webhook/paperless/intake-triage`	Fetches a Paperless document by `document_id`, runs conservative AI triage, writes a Doris Dashboard review queue JSON file, and keeps safe auto-apply disabled unless explicitly enabled
18	Telegram School Intake → Postgres → Paperless v1.2	`POST /webhook/school/intake/upload`	Accepts multipart schoolwork uploads plus class/assignment metadata, uploads into Paperless with deterministic `intake_id` in the filename/title, and returns the resulting task/document context
19	Paperless School Intake Metadata Enrichment v1.1	`POST /webhook/school/intake/paperless-enrich`	Handles Paperless post-consumption webhooks, resolves metadata from deterministic filename/title, then applies deterministic title/tags and optional correspondent mapping

How to Import

Open n8n at https://n8n.paccoco.com
Go to Workflows → click the ⋮ menu → Import from File
Select each .json file from this folder
Activate the workflow (toggle in top-right)

Gotify Channel Map

Each workflow uses a dedicated Gotify app credential. Create these apps in Gotify and add matching credentials in n8n → Credentials:

Gotify App	Used by
Gotify account	01 (Grafana Alerts)
Paperless	03
Whisper	04
Paperless RAG	05
RSS Feed	06
Git Commits	07
Class Recordings	08
Reminders	14
n8n Health	15
Uptime / Infra	16

One-Time Setup

Postgres Dedup Tables (required for workflows 01, 15)

-- Run via: sudo docker exec -it shared-postgres psql -U postgres -d n8n
CREATE TABLE IF NOT EXISTS grafana_alert_dedup (
  fingerprint TEXT PRIMARY KEY,
  last_seen   TIMESTAMPTZ DEFAULT NOW(),
  alert_name  TEXT,
  host        TEXT
);
CREATE TABLE IF NOT EXISTS n8n_health_dedup (
  fingerprint    TEXT PRIMARY KEY,
  last_seen      TIMESTAMPTZ DEFAULT NOW(),
  workflow_names TEXT
);

n8n Environment Variables

Two steps required — both the .env file AND the docker-compose.yaml environment: section must be updated, then n8n restarted with sudo docker compose --env-file .env down n8n && sudo docker compose --env-file .env up -d n8n.

automation/.env — add these:

Variable	Used by	How to get
`PAPERLESS_API_TOKEN`	04, 08, 18, 19	Paperless → Settings → API token
`PAPERLESS_TAG_TRANSCRIPTION`	04	Tag ID from Paperless API
`PAPERLESS_TAG_LECTURE_NOTES`	08	Tag ID from Paperless API
`PAPERLESS_TAG_SCHOOL`	19	Generic school tag ID from Paperless API
`PAPERLESS_TAG_ASSIGNMENTS`	19	Generic assignments tag ID from Paperless API
`PAPERLESS_TAG_TELEGRAM`	19	Tag ID for Telegram-submitted items
`PAPERLESS_TAG_CLASS_MAP_JSON`	19	JSON object mapping class name → Paperless tag ID
`PAPERLESS_TAG_SUBMISSION_KIND_MAP_JSON`	19	JSON object mapping submission kind → Paperless tag ID
`PAPERLESS_DOCUMENT_TYPE_PAPER_KIND_MAP_JSON`	19	JSON object mapping paper kind → Paperless document type ID
`PAPERLESS_CORRESPONDENT_CLASS_MAP_JSON`	19	JSON object mapping class name → Paperless correspondent ID
`PAPERLESS_TRIAGE_MODEL`	17	Optional LiteLLM model override; defaults to `medium`
`PAPERLESS_TRIAGE_REVIEW_QUEUE_PATH`	17	Writable queue path inside the n8n container, e.g. `/data/paperless-triage/paperless_review.json`
`PAPERLESS_TRIAGE_REVIEW_MAX_ITEMS`	17	Optional queue size cap; defaults to `200`
`PAPERLESS_TRIAGE_APPLY_SAFE`	17	Keep `false` unless you intentionally want title-only safe auto-apply enabled
`N8N_API_KEY`	15	n8n → Settings → API → Create API Key
`LITELLM_API_KEY`	01, 04, 08	LiteLLM virtual key (already in .env)

automation/docker-compose.yaml — add to n8n environment: section:

      N8N_BLOCK_ENV_ACCESS_IN_NODE: "false"
      LITELLM_API_KEY: ${LITELLM_API_KEY}
      PAPERLESS_API_TOKEN: ${PAPERLESS_API_TOKEN}
      PAPERLESS_TAG_TRANSCRIPTION: ${PAPERLESS_TAG_TRANSCRIPTION}
      PAPERLESS_TAG_LECTURE_NOTES: ${PAPERLESS_TAG_LECTURE_NOTES}
      PAPERLESS_TAG_SCHOOL: ${PAPERLESS_TAG_SCHOOL}
      PAPERLESS_TAG_ASSIGNMENTS: ${PAPERLESS_TAG_ASSIGNMENTS}
      PAPERLESS_TAG_TELEGRAM: ${PAPERLESS_TAG_TELEGRAM}
      PAPERLESS_TAG_CLASS_MAP_JSON: ${PAPERLESS_TAG_CLASS_MAP_JSON}
      PAPERLESS_TAG_SUBMISSION_KIND_MAP_JSON: ${PAPERLESS_TAG_SUBMISSION_KIND_MAP_JSON}
      PAPERLESS_DOCUMENT_TYPE_PAPER_KIND_MAP_JSON: ${PAPERLESS_DOCUMENT_TYPE_PAPER_KIND_MAP_JSON}
      PAPERLESS_CORRESPONDENT_CLASS_MAP_JSON: ${PAPERLESS_CORRESPONDENT_CLASS_MAP_JSON}
      PAPERLESS_TRIAGE_MODEL: ${PAPERLESS_TRIAGE_MODEL}
      PAPERLESS_TRIAGE_REVIEW_QUEUE_PATH: ${PAPERLESS_TRIAGE_REVIEW_QUEUE_PATH}
      PAPERLESS_TRIAGE_REVIEW_MAX_ITEMS: ${PAPERLESS_TRIAGE_REVIEW_MAX_ITEMS}
      PAPERLESS_TRIAGE_APPLY_SAFE: ${PAPERLESS_TRIAGE_APPLY_SAFE}
      N8N_API_KEY: ${N8N_API_KEY}

Note: N8N_BLOCK_ENV_ACCESS_IN_NODE: "false" is required for workflows to read $env.* variables. Without it, env var access is silently denied even if the variable exists in the container.

SSH Credential (required for workflow 16)

Workflow 16 SSHes into PD to probe mounts and restart containers. Set up key-based auth:

# Run on PD — generate a dedicated key for n8n
ssh-keygen -t ed25519 -f ~/.ssh/n8n_watchdog -N ""
cat ~/.ssh/n8n_watchdog.pub >> ~/.ssh/authorized_keys
cat ~/.ssh/n8n_watchdog   # paste this into n8n → Credentials → SSH (Private Key)

In n8n: Credentials → New → SSH (Private Key) → host 10.5.30.6, user truenas_admin, paste private key. Name it PD SSH.

Workflow 16 also runs sudo docker restart and the mount script over SSH. Add a NOPASSWD sudoers rule so it doesn't hang:

sudo visudo -f /etc/sudoers.d/n8n-watchdog
# Add:
# truenas_admin ALL=(ALL) NOPASSWD: /usr/bin/docker, /bin/bash /mnt/tank/docker/scripts/mount-unraid-nfs.sh

Qdrant Collection (required for workflows 02, 05, 08)

Create the knowledge_base collection before using the RAG workflows:

curl -X PUT http://10.5.30.6:6333/collections/knowledge_base \
  -H 'Content-Type: application/json' \
  -d '{
    "vectors": {
      "size": 768,
      "distance": "Cosine"
    }
  }'

(768 dimensions matches nomic-embed-text. If you use a different embedding model, adjust the size.)

Pull nomic-embed-text into Ollama (required for workflows 02, 05, 08)

curl http://10.5.30.6:11434/api/pull -d '{"name": "nomic-embed-text"}'

Paperless Webhook (required for workflows 03, 05, 17, and 19)

In Paperless-NGX, set up post-consumption webhooks to POST to:

https://n8n.paccoco.com/webhook/paperless/new-document
https://n8n.paccoco.com/webhook/paperless/rag-ingest
https://n8n.paccoco.com/webhook/paperless/intake-triage
https://n8n.paccoco.com/webhook/school/intake/paperless-enrich

with body: {"document_id": <id>}

You can trigger all four from the same Paperless event. Workflow 03 handles general AI summary/tagging, workflow 05 handles RAG ingest, workflow 17 feeds the Doris Dashboard review queue, and workflow 19 re-applies deterministic school metadata for intake-managed documents.

Grafana Webhook (already configured)

Your Grafana contact point is already set to https://n8n.paccoco.com/webhook/grafana-alert — workflow 01 will work immediately once activated.

Git Webhook (required for workflow 07)

In your Gitea/Forgejo/GitHub repo settings, add a webhook:

URL: https://n8n.paccoco.com/webhook/git/push
Content type: application/json
Events: Push only

Works with Gitea, Forgejo, GitHub, and GitLab webhook payloads.

Usage Examples

Upload schoolwork from Telegram intake:

curl -X POST https://n8n.paccoco.com/webhook/school/intake/upload \
  -F "document=@essay-draft.pdf" \
  -F "class_name=English 101" \
  -F "assignment_name=Essay Draft 1" \
  -F "submission_kind=homework" \
  -F "semester=Fall 2026" \
  -F "course_code=ENG101" \
  -F "paper_kind=essay"

Ingest a document into RAG:

curl -X POST https://n8n.paccoco.com/webhook/rag/ingest \
  -H 'Content-Type: application/json' \
  -d '{
    "text": "Your document content here...",
    "source": "my-notes",
    "metadata": {"topic": "homelab"}
  }'

Query your knowledge base:

curl -X POST https://n8n.paccoco.com/webhook/rag/query \
  -H 'Content-Type: application/json' \
  -d '{"query": "How do I configure Qdrant?"}'

Transcribe audio:

curl -X POST https://n8n.paccoco.com/webhook/transcribe \
  -F "data=@recording.mp3"

Transcribe + AI summarize:

curl -X POST https://n8n.paccoco.com/webhook/transcribe/summarize \
  -F "data=@meeting.mp3"

Add a class recording (just drop a file):

/mnt/tank/class-recordings/
├── CS101-Java/
│   ├── lecture-01-intro.wav
│   ├── lecture-02-variables.wav
│   └── lecture-03-loops.wav
├── MATH201/
│   └── lecture-01-derivatives.wav
└── ENG102/
    └── essay-workshop.wav

Workflow 08 uses a webhook trigger (POST /webhook/class/upload), not a folder watcher. Upload the file via curl and include the class name and lecture title as form fields (see curl example below).

Note: The folder structure above shows suggested organization on disk. The workflow does not watch the folder — you must POST to the webhook.

Query RAG for class-specific content:

curl -X POST https://n8n.paccoco.com/webhook/rag/query \
  -H 'Content-Type: application/json' \
  -d '{"query": "What did my CS101 lecture say about inheritance in Java?"}'

The RAG system stores class name metadata with each chunk, so when you mention a class name in your query it will prioritize results from that class.

Class Recordings Setup (Workflow 08)

Create class subfolders on PD (already done for CS101-Java):

sudo mkdir -p /mnt/tank/class-recordings/CS101-Java
sudo mkdir -p /mnt/tank/class-recordings/MATH201

Add to n8n's automation/.env:

PAPERLESS_TAG_LECTURE_NOTES=11

Add to n8n's automation/docker-compose.yaml under the n8n environment: section:

      PAPERLESS_TAG_LECTURE_NOTES: ${PAPERLESS_TAG_LECTURE_NOTES}

Then restart n8n and import/activate 08-class-recording-rag-v1.1.json.

To upload a class recording:

curl -X POST https://n8n.paccoco.com/webhook/class/upload \
  -F "data=@lecture-01-intro.wav" \
  -F "class_name=CS101-Java" \
  -F "lecture_title=lecture-01-intro"

Trigger RSS digest manually:

curl -X POST https://n8n.paccoco.com/webhook/scrape/summarize

Test git webhook:

curl -X POST https://n8n.paccoco.com/webhook/git/push \
  -H 'Content-Type: application/json' \
  -d '{
    "ref": "refs/heads/main",
    "pusher": {"name": "john"},
    "repository": {"full_name": "john/my-project"},
    "commits": [
      {"id": "abc1234", "message": "Fix null pointer in UserService", "author": {"name": "john"}, "added": [], "modified": ["src/UserService.java"], "removed": []}
    ]
  }'

Workflow 18 — Telegram School Intake → Postgres → Paperless v1.2

Import 18-school-paperless-intake-v1.2.json for a chat-driven schoolwork intake flow. Supporting notes live at docs/operations/SCHOOL_PAPERLESS_INTAKE.md and the tracked schema lives at docs/reference/SCHOOL_INTAKE_POSTGRES_SCHEMA.sql.

Highlights:

multipart upload endpoint: POST /webhook/school/intake/upload
required fields: class_name, assignment_name, submission_kind
optional fields: semester, course_code, paper_kind, notes, telegram_chat_id, telegram_message_id
sends deterministic intake_id in the Paperless filename and title correlation data
returns upload success context immediately from the webhook
v1.1 removed the require('crypto') dependency so the workflow can run without adding crypto to NODE_FUNCTION_ALLOW_BUILTIN
v1.2 fixes the multipart Paperless upload node to use n8n's correct inputDataFieldName pattern for binary form uploads

Testing fixture:

fixtures/school-paperless-intake-webhook.curl.example.txt

Workflow Versioning Convention

For n8n workflow artifacts in this repo:

include the version in the filename (example: 18-school-paperless-intake-v1.1.json)
include the version in the workflow's internal name field
bump the version whenever behavior, schema expectations, webhook payload handling, or env requirements change
update n8n-workflows/README.md and n8n-workflows/TESTING-STATUS.md when introducing a new workflow version

Do not overwrite a workflow artifact with silent behavior changes and no version bump. That way lies confusion and sweaty guesswork. Ha ha.

Customizing RSS Feeds (Workflow 06)

Edit the "Define RSS Feeds" Code node to add/remove feeds. Default feeds:

Self-Hosted Show
r/selfhosted
r/homelab
Hacker News

Add any RSS feed URL you want monitored.

Workflow 17 — Paperless Intake Triage → Doris Review Queue

Import 17-paperless-intake-triage.json for a review-first Paperless intake workflow. It is intentionally conservative: it writes a sanitized local review queue for Doris Dashboard and does not delete documents. Automatic Paperless updates are disabled by default.

Trigger

Configure a Paperless-NGX post-consumption webhook to POST to:

https://n8n.paccoco.com/webhook/paperless/intake-triage

Body:

{"document_id": 12345}

Fixture: fixtures/paperless-intake-webhook.example.json.

Environment variables

Add these to automation/.env and expose them in the n8n environment: block:

Variable	Required	Default	Purpose
`PAPERLESS_BASE_URL`	yes	`https://paperless.paccoco.com`	Paperless API/UI base URL
`PAPERLESS_API_TOKEN`	yes	none	Paperless API token; used as `Token ...`
`LITELLM_BASE_URL`	yes	`http://10.5.30.6:4000`	LiteLLM OpenAI-compatible endpoint
`LITELLM_API_KEY`	yes	none	LiteLLM key
`PAPERLESS_TRIAGE_MODEL`	no	`gpt-4o-mini`	Model for strict JSON triage
`PAPERLESS_TRIAGE_REVIEW_QUEUE_PATH`	yes	`/data/paperless_review.json`	Sanitized JSON queue path mounted for Doris Dashboard
`PAPERLESS_TRIAGE_REVIEW_MAX_ITEMS`	no	`200`	Max queue entries retained
`PAPERLESS_TRIAGE_APPLY_SAFE`	no	`false`	Enables low-risk title-only update branch when `true`
`GOTIFY_URL`	no	none	Enables urgent-doc notification only when token also set
`GOTIFY_TOKEN`	no	none	Gotify app token for urgent docs
`NODE_FUNCTION_ALLOW_BUILTIN`	yes for queue file	none	Must include `fs,path` for the Code node that writes JSON
`N8N_BLOCK_ENV_ACCESS_IN_NODE`	yes	existing note	Must be `false` so Code nodes can read `$env`

Example environment additions:

      N8N_BLOCK_ENV_ACCESS_IN_NODE: "false"
      NODE_FUNCTION_ALLOW_BUILTIN: fs,path
      PAPERLESS_BASE_URL: ${PAPERLESS_BASE_URL}
      PAPERLESS_API_TOKEN: ${PAPERLESS_API_TOKEN}
      LITELLM_BASE_URL: ${LITELLM_BASE_URL}
      LITELLM_API_KEY: ${LITELLM_API_KEY}
      PAPERLESS_TRIAGE_MODEL: ${PAPERLESS_TRIAGE_MODEL}
      PAPERLESS_TRIAGE_REVIEW_QUEUE_PATH: /workspace/doris-dashboard/data/paperless_review.json
      PAPERLESS_TRIAGE_APPLY_SAFE: "false"

Mount the Doris dashboard data directory read/write into the n8n container if it is not already mounted. The workflow writes only sanitized display data: document IDs, titles, filenames, archive URL, current metadata names/IDs, triage summary, review reason, urgency, confidence, and safety flags. It must not contain tokens.

Safety gates

The LLM must return strict JSON:

{
  "summary": "plain English summary",
  "suggested_title": "clean title",
  "document_type": "bill|receipt|school|medical|tax|insurance|legal|manual|letter|other",
  "suggested_correspondent": "string or null",
  "suggested_tags": ["..."],
  "due_date": "YYYY-MM-DD or null",
  "needs_review": true,
  "urgency": "none|low|medium|high",
  "reason": "why review/why not",
  "confidence": "high|medium|low"
}

The workflow forcibly marks needs_review=true for:

legal, medical, school, tax, or insurance documents
any due date
bill/payment/invoice language
low confidence
urgency medium or high

Only high-confidence, low-risk docs with no due date and urgency none|low become eligible for automatic update, and even then the branch is disabled unless PAPERLESS_TRIAGE_APPLY_SAFE=true. The safe update branch currently updates title only. It does not delete, archive, or apply high-risk metadata.

Test payload and expected result

Webhook payload: fixtures/paperless-intake-webhook.example.json
Example Paperless API document: fixtures/paperless-intake-document.example.json
Expected triage JSON: fixtures/paperless-intake-expected-triage.example.json

The fixture is a utility bill with a due date, so the safety gate must force human review.

Import/deploy steps

Import 17-paperless-intake-triage.json in n8n.
Add env vars and required volume mount.
Restart n8n using the normal automation compose procedure.
Keep PAPERLESS_TRIAGE_APPLY_SAFE=false for initial testing.
Use a known test document and POST {"document_id": <id>} to /webhook/paperless/intake-triage.
Confirm doris-dashboard/data/paperless_review.json updates and contains no secrets.
Activate the Paperless webhook only after the test queue file is correct.

Rollback

Deactivate the workflow and remove the Paperless webhook target. If needed, move doris-dashboard/data/paperless_review.json aside; it is display-only derived data. No Paperless deletion path exists in this workflow.

20 KiB Raw Blame History