feat(n8n): add school paperless intake pipeline
This commit is contained in:
@@ -3,3 +3,15 @@ TZ=America/Chicago
|
||||
N8N_HOST=n8n.paccoco.com
|
||||
N8N_DB_PASS=CHANGE_ME
|
||||
N8N_ENCRYPTION_KEY=CHANGE_ME
|
||||
PAPERLESS_BASE_URL=https://paperless.paccoco.com
|
||||
PAPERLESS_API_TOKEN=CHANGE_ME
|
||||
PAPERLESS_CORRESPONDENT_SCHOOL=
|
||||
PAPERLESS_DOCUMENT_TYPE_SCHOOL=
|
||||
PAPERLESS_TAG_SCHOOL=
|
||||
PAPERLESS_TAG_ASSIGNMENTS=
|
||||
PAPERLESS_TAG_TELEGRAM=
|
||||
PAPERLESS_TAG_CLASS_MAP_JSON={}
|
||||
PAPERLESS_TAG_SUBMISSION_KIND_MAP_JSON={}
|
||||
PAPERLESS_DOCUMENT_TYPE_PAPER_KIND_MAP_JSON={}
|
||||
PAPERLESS_CORRESPONDENT_CLASS_MAP_JSON={}
|
||||
NODE_FUNCTION_ALLOW_BUILTIN=crypto
|
||||
|
||||
@@ -40,6 +40,18 @@ services:
|
||||
EXECUTIONS_DATA_MAX_AGE: 336
|
||||
N8N_BLOCK_ENV_ACCESS_IN_NODE: "false"
|
||||
LITELLM_API_KEY: ${LITELLM_API_KEY}
|
||||
PAPERLESS_BASE_URL: ${PAPERLESS_BASE_URL}
|
||||
PAPERLESS_API_TOKEN: ${PAPERLESS_API_TOKEN}
|
||||
PAPERLESS_CORRESPONDENT_SCHOOL: ${PAPERLESS_CORRESPONDENT_SCHOOL}
|
||||
PAPERLESS_DOCUMENT_TYPE_SCHOOL: ${PAPERLESS_DOCUMENT_TYPE_SCHOOL}
|
||||
PAPERLESS_TAG_SCHOOL: ${PAPERLESS_TAG_SCHOOL}
|
||||
PAPERLESS_TAG_ASSIGNMENTS: ${PAPERLESS_TAG_ASSIGNMENTS}
|
||||
PAPERLESS_TAG_TELEGRAM: ${PAPERLESS_TAG_TELEGRAM}
|
||||
PAPERLESS_TAG_CLASS_MAP_JSON: ${PAPERLESS_TAG_CLASS_MAP_JSON}
|
||||
PAPERLESS_TAG_SUBMISSION_KIND_MAP_JSON: ${PAPERLESS_TAG_SUBMISSION_KIND_MAP_JSON}
|
||||
PAPERLESS_DOCUMENT_TYPE_PAPER_KIND_MAP_JSON: ${PAPERLESS_DOCUMENT_TYPE_PAPER_KIND_MAP_JSON}
|
||||
PAPERLESS_CORRESPONDENT_CLASS_MAP_JSON: ${PAPERLESS_CORRESPONDENT_CLASS_MAP_JSON}
|
||||
NODE_FUNCTION_ALLOW_BUILTIN: ${NODE_FUNCTION_ALLOW_BUILTIN}
|
||||
volumes:
|
||||
- /mnt/tank/docker/appdata/n8n:/home/node/.n8n
|
||||
- /mnt/data/class-recordings:/data/class-recordings
|
||||
|
||||
111
docs/operations/SCHOOL_PAPERLESS_INTAKE.md
Normal file
111
docs/operations/SCHOOL_PAPERLESS_INTAKE.md
Normal file
@@ -0,0 +1,111 @@
|
||||
# School Paperless Intake Workflow
|
||||
|
||||
Workflows:
|
||||
- `n8n-workflows/18-school-paperless-intake.json`
|
||||
- `n8n-workflows/19-paperless-school-metadata-enrichment.json`
|
||||
|
||||
This workflow is meant for a Telegram chat-driven upload pipeline that sends schoolwork into Paperless-NGX while recording intake metadata in shared Postgres.
|
||||
|
||||
## What it does
|
||||
|
||||
1. Accepts a multipart upload at `POST /webhook/school/intake/upload`
|
||||
2. Requires form fields:
|
||||
- `class_name`
|
||||
- `assignment_name`
|
||||
- `submission_kind`
|
||||
3. Accepts optional fields:
|
||||
- `semester`
|
||||
- `course_code`
|
||||
- `paper_kind`
|
||||
- `notes`
|
||||
- `source` (defaults to `telegram`)
|
||||
- `telegram_chat_id`
|
||||
- `telegram_message_id`
|
||||
4. Generates a deterministic `intake_id` from date + class + assignment + file checksum
|
||||
5. Stores/updates the intake row in Postgres first
|
||||
6. Uploads the file to Paperless using the deterministic intake ID in the filename
|
||||
7. Marks the row as `uploaded` after Paperless accepts the upload
|
||||
|
||||
## Shared Postgres expectations
|
||||
|
||||
- Use the shared Postgres network/credentials pattern already used by the automation stack.
|
||||
- The workflow expects an n8n Postgres credential named `Shared Postgres`.
|
||||
- Recommended target database: `school_intake`
|
||||
- Tracked schema: `docs/reference/SCHOOL_INTAKE_POSTGRES_SCHEMA.sql`
|
||||
|
||||
## Required environment variables
|
||||
|
||||
Add to `automation/.env`:
|
||||
|
||||
```env
|
||||
PAPERLESS_BASE_URL=https://paperless.paccoco.com
|
||||
PAPERLESS_API_TOKEN=CHANGE_ME
|
||||
LITELLM_API_KEY=CHANGE_ME
|
||||
PAPERLESS_CORRESPONDENT_SCHOOL=
|
||||
PAPERLESS_DOCUMENT_TYPE_SCHOOL=
|
||||
PAPERLESS_TAG_SCHOOL=
|
||||
PAPERLESS_TAG_ASSIGNMENTS=
|
||||
PAPERLESS_TAG_TELEGRAM=
|
||||
NODE_FUNCTION_ALLOW_BUILTIN=crypto
|
||||
```
|
||||
|
||||
Add to the `n8n` service `environment:` block in `automation/docker-compose.yaml`:
|
||||
|
||||
```yaml
|
||||
PAPERLESS_BASE_URL: ${PAPERLESS_BASE_URL}
|
||||
PAPERLESS_API_TOKEN: ${PAPERLESS_API_TOKEN}
|
||||
PAPERLESS_CORRESPONDENT_SCHOOL: ${PAPERLESS_CORRESPONDENT_SCHOOL}
|
||||
PAPERLESS_DOCUMENT_TYPE_SCHOOL: ${PAPERLESS_DOCUMENT_TYPE_SCHOOL}
|
||||
PAPERLESS_TAG_SCHOOL: ${PAPERLESS_TAG_SCHOOL}
|
||||
PAPERLESS_TAG_ASSIGNMENTS: ${PAPERLESS_TAG_ASSIGNMENTS}
|
||||
PAPERLESS_TAG_TELEGRAM: ${PAPERLESS_TAG_TELEGRAM}
|
||||
NODE_FUNCTION_ALLOW_BUILTIN: crypto
|
||||
```
|
||||
|
||||
Notes:
|
||||
- `PAPERLESS_CORRESPONDENT_SCHOOL` and `PAPERLESS_DOCUMENT_TYPE_SCHOOL` should be numeric IDs if you want those fields auto-assigned.
|
||||
- Leave optional Paperless IDs blank if you prefer Paperless rules to classify later.
|
||||
- `NODE_FUNCTION_ALLOW_BUILTIN=crypto` is required because the Code node hashes the uploaded file.
|
||||
- `N8N_BLOCK_ENV_ACCESS_IN_NODE` must remain `false`.
|
||||
|
||||
## Paperless metadata behavior
|
||||
|
||||
The upload uses:
|
||||
- filename: `<intake_id>.<ext>`
|
||||
- title: `<class_name> — <assignment_name> — <submission_kind>`
|
||||
|
||||
Workflow 18 deliberately keeps the Paperless upload minimal and safe:
|
||||
- multipart `document`
|
||||
- deterministic title
|
||||
|
||||
The deterministic filename/title preserve the `intake_id` for later correlation.
|
||||
|
||||
Workflow 19 handles the follow-up metadata pass after Paperless finishes processing:
|
||||
- fetch Paperless document by webhook document id
|
||||
- resolve `intake_id` from the stored filename
|
||||
- look up the intake record in shared Postgres
|
||||
- re-apply deterministic title
|
||||
- merge generic school tags plus optional class/submission-kind mappings
|
||||
- optionally set document type / correspondent from JSON env mappings
|
||||
|
||||
## Testing
|
||||
|
||||
Use the example curl request in:
|
||||
- `n8n-workflows/fixtures/school-paperless-intake-webhook.curl.example.txt`
|
||||
|
||||
Minimum test checklist:
|
||||
1. Import the workflow into n8n.
|
||||
2. Attach the `Shared Postgres` credential.
|
||||
3. Confirm the target DB/table exists.
|
||||
4. Send the sample multipart request.
|
||||
5. Verify a row appears in `school_paperless_intake` with `status=uploaded`.
|
||||
6. Verify the uploaded Paperless document title and stored filename both include the deterministic intake ID.
|
||||
7. Trigger workflow 19 with the Paperless document id and verify the row moves to `status=enriched`.
|
||||
8. Re-submit the same file and metadata to confirm the same `intake_id` is reused and the Postgres row is upserted.
|
||||
|
||||
## Known assumptions / caveats
|
||||
|
||||
- The workflow is repo-only and was not live-tested against the running n8n/Paperless stack.
|
||||
- The Postgres node uses SQL expressions inline rather than parameter binding because exported n8n node JSON can differ by version.
|
||||
- If your n8n build blocks `require('crypto')`, add `NODE_FUNCTION_ALLOW_BUILTIN=crypto` to the container env.
|
||||
- If Paperless needs tags as repeated `tags[]` fields instead of a comma-separated `tags` field, adjust the HTTP Request node after import.
|
||||
85
docs/planning/SCHOOL_INTAKE_PIPELINE.md
Normal file
85
docs/planning/SCHOOL_INTAKE_PIPELINE.md
Normal file
@@ -0,0 +1,85 @@
|
||||
# School Work Intake Pipeline (Telegram → Postgres → Paperless → n8n)
|
||||
|
||||
## Goal
|
||||
|
||||
Give Doris a reliable way to ingest school work from Telegram while preserving explicit metadata:
|
||||
- class
|
||||
- assignment
|
||||
- submission kind (`draft`, `final`, etc.)
|
||||
- optional semester / instructor / due date / paper kind
|
||||
|
||||
The design must be expandable for new classes and future non-paper workflows.
|
||||
|
||||
## Recommended architecture
|
||||
|
||||
1. **Telegram chat-driven intake**
|
||||
- John sends file(s) to Doris on Telegram.
|
||||
- Doris asks follow-up questions until required metadata is complete.
|
||||
|
||||
2. **n8n intake webhook**
|
||||
- Doris submits the file + metadata to an intake webhook.
|
||||
- The webhook creates a durable intake record in shared Postgres.
|
||||
- The webhook stages/uploads the file into Paperless with a deterministic `intake_id` embedded in the filename.
|
||||
|
||||
3. **Paperless post-consumption webhook**
|
||||
- After Paperless finishes processing the file, a second workflow fetches the Paperless document.
|
||||
- That workflow extracts `intake_id`, looks up the Postgres intake record, and applies metadata.
|
||||
|
||||
4. **Metadata + AI enrichment**
|
||||
- Deterministic metadata first: class tags, version tags, title rules, optional correspondent/document type IDs.
|
||||
- AI second: document summary/note and any low-risk enrichment.
|
||||
- Ambiguous cases can be routed to a review queue.
|
||||
|
||||
## Why this is better than folder-based consume ingestion
|
||||
|
||||
Folder paths are a decent hint, but they are not durable enough to be the only source of truth.
|
||||
|
||||
Problems with folder-only inference:
|
||||
- folder context can disappear after import
|
||||
- mixed uploads are easy to mislabel
|
||||
- retries/reprocessing can grab the wrong document
|
||||
- future use-cases (draft review, assignment history, instructor-specific rules) need explicit metadata anyway
|
||||
|
||||
## Shared DB requirements
|
||||
|
||||
Use shared Postgres, not SQLite.
|
||||
|
||||
Schema artifact:
|
||||
- `docs/reference/SCHOOL_INTAKE_POSTGRES_SCHEMA.sql`
|
||||
|
||||
Core tables:
|
||||
- `school_paperless_intake`
|
||||
- `school_paperless_intake_events`
|
||||
|
||||
## Config-driven class profiles
|
||||
|
||||
Class/course behavior should live in config, not hardcoded workflow branches.
|
||||
|
||||
Suggested profile fields:
|
||||
- `class_key`
|
||||
- `display_name`
|
||||
- `course_code`
|
||||
- `default_tags`
|
||||
- Paperless IDs for correspondent/document type/storage path when needed
|
||||
- title template override
|
||||
|
||||
## Initial rollout target
|
||||
|
||||
- ENGL-1010
|
||||
- HIST-2020
|
||||
|
||||
## OpenClaw helper artifact
|
||||
|
||||
Local helper:
|
||||
- `school/intake/submit_to_n8n.js`
|
||||
|
||||
This gives Doris a simple handoff point after the Telegram conversation is complete.
|
||||
|
||||
## Live work still needed
|
||||
|
||||
- import/activate the new n8n intake workflow
|
||||
- import/activate the Paperless enrichment workflow
|
||||
- create shared Postgres database/user/schema
|
||||
- configure class profiles with real Paperless IDs
|
||||
- decide Telegram delivery wording/confirmation behavior
|
||||
- test with real ENGL + HIST documents
|
||||
@@ -41,6 +41,9 @@
|
||||
- [ ] Add sudoers NOPASSWD rule for n8n watchdog (see n8n-workflows/README.md)
|
||||
- [ ] Import and activate upgraded workflows: 04, 08 (01 done — v1.4 active and tested)
|
||||
- [ ] Import and activate new workflows: 14, 15 (16 already active)
|
||||
- [ ] Build Telegram chat-driven school-work intake for Paperless using shared Postgres, deterministic `intake_id`, and class/version metadata
|
||||
- [ ] Configure class profiles + Paperless metadata mapping for current courses (ENGL-1010, HIST-2020, future classes)
|
||||
- [ ] Set up DB and Docker backups to Serenity following homelab SOP
|
||||
|
||||
## Completed
|
||||
- [x] Deploy Gotify (Phase 1) — 2026-05-05
|
||||
|
||||
47
docs/reference/SCHOOL_INTAKE_POSTGRES_SCHEMA.sql
Normal file
47
docs/reference/SCHOOL_INTAKE_POSTGRES_SCHEMA.sql
Normal file
@@ -0,0 +1,47 @@
|
||||
-- Shared Postgres schema for Telegram-driven school-work intake.
|
||||
-- Apply in a dedicated `school_intake` database or adapt to an existing shared DB.
|
||||
|
||||
CREATE TABLE IF NOT EXISTS school_paperless_intake (
|
||||
intake_id TEXT PRIMARY KEY,
|
||||
class_name TEXT NOT NULL,
|
||||
assignment_name TEXT NOT NULL,
|
||||
submission_kind TEXT NOT NULL,
|
||||
semester TEXT,
|
||||
course_code TEXT,
|
||||
paper_kind TEXT,
|
||||
notes TEXT,
|
||||
source TEXT NOT NULL DEFAULT 'telegram',
|
||||
telegram_chat_id TEXT,
|
||||
telegram_message_id TEXT,
|
||||
original_filename TEXT NOT NULL,
|
||||
stored_filename TEXT NOT NULL,
|
||||
mime_type TEXT NOT NULL,
|
||||
file_size_bytes BIGINT NOT NULL,
|
||||
checksum_sha256 TEXT NOT NULL,
|
||||
status TEXT NOT NULL DEFAULT 'received',
|
||||
paperless_task_id TEXT,
|
||||
paperless_document_id BIGINT,
|
||||
paperless_title TEXT,
|
||||
received_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
|
||||
uploaded_at TIMESTAMPTZ,
|
||||
enriched_at TIMESTAMPTZ,
|
||||
error_text TEXT,
|
||||
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
|
||||
updated_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
|
||||
);
|
||||
|
||||
CREATE INDEX IF NOT EXISTS idx_school_paperless_intake_status ON school_paperless_intake(status);
|
||||
CREATE INDEX IF NOT EXISTS idx_school_paperless_intake_doc_id ON school_paperless_intake(paperless_document_id);
|
||||
CREATE INDEX IF NOT EXISTS idx_school_paperless_intake_assignment ON school_paperless_intake(class_name, assignment_name);
|
||||
CREATE INDEX IF NOT EXISTS idx_school_paperless_intake_checksum ON school_paperless_intake(checksum_sha256);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS school_paperless_intake_events (
|
||||
id BIGSERIAL PRIMARY KEY,
|
||||
intake_id TEXT NOT NULL REFERENCES school_paperless_intake(intake_id) ON DELETE CASCADE,
|
||||
event_type TEXT NOT NULL,
|
||||
event_payload JSONB NOT NULL DEFAULT '{}'::jsonb,
|
||||
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
|
||||
);
|
||||
|
||||
CREATE INDEX IF NOT EXISTS idx_school_paperless_intake_events_intake_id
|
||||
ON school_paperless_intake_events(intake_id);
|
||||
237
n8n-workflows/18-school-paperless-intake.json
Normal file
237
n8n-workflows/18-school-paperless-intake.json
Normal file
@@ -0,0 +1,237 @@
|
||||
{
|
||||
"name": "Telegram School Intake → Postgres → Paperless",
|
||||
"active": false,
|
||||
"isArchived": false,
|
||||
"nodes": [
|
||||
{
|
||||
"parameters": {
|
||||
"httpMethod": "POST",
|
||||
"path": "school/intake/upload",
|
||||
"responseMode": "lastNode",
|
||||
"responseData": "allEntries",
|
||||
"options": {
|
||||
"rawBody": true
|
||||
}
|
||||
},
|
||||
"id": "school-intake-webhook",
|
||||
"name": "School Intake Webhook",
|
||||
"type": "n8n-nodes-base.webhook",
|
||||
"typeVersion": 2,
|
||||
"position": [
|
||||
-1180,
|
||||
0
|
||||
]
|
||||
},
|
||||
{
|
||||
"parameters": {
|
||||
"jsCode": "\nconst crypto = require('crypto');\nconst req = $input.first().json;\nconst body = req.body || {};\nconst binary = $input.first().binary || {};\nconst upload = binary.data || Object.values(binary)[0];\nif (!upload) throw new Error('Expected multipart file field named \\\"data\\\".');\nconst required = ['class_name', 'assignment_name', 'submission_kind'];\nfor (const key of required) {\n if (!body[key] || !String(body[key]).trim()) {\n throw new Error('Missing required form field: ' + key + '. Required fields: class_name, assignment_name, submission_kind.');\n }\n}\nconst sanitize = (value, fallback = 'item') => String(value || fallback)\n .trim()\n .toLowerCase()\n .replace(/[^a-z0-9]+/g, '-')\n .replace(/^-+|-+$/g, '')\n .slice(0, 80) || fallback;\nconst originalName = upload.fileName || body.filename || 'upload.bin';\nconst extension = (() => {\n const m = String(originalName).match(/(\\.[A-Za-z0-9]{1,10})$/);\n return m ? m[1].toLowerCase() : '';\n})();\nconst fileBytes = Buffer.from(upload.data, 'base64');\nconst checksum = crypto.createHash('sha256').update(fileBytes).digest('hex');\nconst now = new Date();\nconst intakeId = [\n 'school',\n now.toISOString().slice(0,10).replace(/-/g,''),\n sanitize(body.class_name),\n sanitize(body.assignment_name),\n checksum.slice(0, 12)\n].join('-');\nconst storedFilename = intakeId + extension;\nconst title = [body.class_name, body.assignment_name, body.submission_kind].map(v => String(v).trim()).join(' — ');\nreturn [{\n json: {\n intake_id: intakeId,\n class_name: String(body.class_name).trim(),\n assignment_name: String(body.assignment_name).trim(),\n submission_kind: String(body.submission_kind).trim(),\n semester: body.semester ? String(body.semester).trim() : null,\n course_code: body.course_code ? String(body.course_code).trim() : null,\n paper_kind: body.paper_kind ? String(body.paper_kind).trim() : null,\n notes: body.notes ? String(body.notes).trim().slice(0, 4000) : null,\n source: body.source ? String(body.source).trim() : 'telegram',\n telegram_chat_id: body.telegram_chat_id ? String(body.telegram_chat_id).trim() : null,\n telegram_message_id: body.telegram_message_id ? String(body.telegram_message_id).trim() : null,\n original_filename: originalName,\n stored_filename: storedFilename,\n mime_type: upload.mimeType || 'application/octet-stream',\n file_size_bytes: fileBytes.length,\n checksum_sha256: checksum,\n paperless_title: title,\n received_at: now.toISOString()\n },\n binary: {\n data: {\n ...upload,\n fileName: storedFilename,\n mimeType: upload.mimeType || 'application/octet-stream'\n }\n }\n}];"
|
||||
},
|
||||
"id": "normalize-request",
|
||||
"name": "Normalize Intake Request",
|
||||
"type": "n8n-nodes-base.code",
|
||||
"typeVersion": 2,
|
||||
"position": [
|
||||
-920,
|
||||
0
|
||||
]
|
||||
},
|
||||
{
|
||||
"parameters": {
|
||||
"operation": "executeQuery",
|
||||
"query": "INSERT INTO school_paperless_intake (\n intake_id,\n class_name,\n assignment_name,\n submission_kind,\n semester,\n course_code,\n paper_kind,\n notes,\n source,\n telegram_chat_id,\n telegram_message_id,\n original_filename,\n stored_filename,\n mime_type,\n file_size_bytes,\n checksum_sha256,\n status,\n received_at\n)\nVALUES (\n '{{ $json.intake_id }}',\n '{{ $json.class_name.replace(/'/g, \"''\") }}',\n '{{ $json.assignment_name.replace(/'/g, \"''\") }}',\n '{{ $json.submission_kind.replace(/'/g, \"''\") }}',\n {{ $json.semester ? \"'\" + $json.semester.replace(/'/g, \"''\") + \"'\" : 'NULL' }},\n {{ $json.course_code ? \"'\" + $json.course_code.replace(/'/g, \"''\") + \"'\" : 'NULL' }},\n {{ $json.paper_kind ? \"'\" + $json.paper_kind.replace(/'/g, \"''\") + \"'\" : 'NULL' }},\n {{ $json.notes ? \"'\" + $json.notes.replace(/'/g, \"''\") + \"'\" : 'NULL' }},\n '{{ $json.source.replace(/'/g, \"''\") }}',\n {{ $json.telegram_chat_id ? \"'\" + $json.telegram_chat_id.replace(/'/g, \"''\") + \"'\" : 'NULL' }},\n {{ $json.telegram_message_id ? \"'\" + $json.telegram_message_id.replace(/'/g, \"''\") + \"'\" : 'NULL' }},\n '{{ $json.original_filename.replace(/'/g, \"''\") }}',\n '{{ $json.stored_filename.replace(/'/g, \"''\") }}',\n '{{ $json.mime_type.replace(/'/g, \"''\") }}',\n {{ $json.file_size_bytes }},\n '{{ $json.checksum_sha256 }}',\n 'received',\n '{{ $json.received_at }}'::timestamptz\n)\nON CONFLICT (intake_id) DO UPDATE SET\n class_name = EXCLUDED.class_name,\n assignment_name = EXCLUDED.assignment_name,\n submission_kind = EXCLUDED.submission_kind,\n semester = EXCLUDED.semester,\n course_code = EXCLUDED.course_code,\n paper_kind = EXCLUDED.paper_kind,\n notes = EXCLUDED.notes,\n source = EXCLUDED.source,\n telegram_chat_id = EXCLUDED.telegram_chat_id,\n telegram_message_id = EXCLUDED.telegram_message_id,\n original_filename = EXCLUDED.original_filename,\n stored_filename = EXCLUDED.stored_filename,\n mime_type = EXCLUDED.mime_type,\n file_size_bytes = EXCLUDED.file_size_bytes,\n checksum_sha256 = EXCLUDED.checksum_sha256,\n status = 'received',\n received_at = EXCLUDED.received_at,\n updated_at = NOW();"
|
||||
},
|
||||
"id": "pg-insert-received",
|
||||
"name": "Postgres Upsert Intake Received",
|
||||
"type": "n8n-nodes-base.postgres",
|
||||
"typeVersion": 2.6,
|
||||
"position": [
|
||||
-660,
|
||||
0
|
||||
],
|
||||
"credentials": {
|
||||
"postgres": {
|
||||
"id": "SETUP_REQUIRED",
|
||||
"name": "Shared Postgres"
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"parameters": {
|
||||
"method": "POST",
|
||||
"url": "={{($env.PAPERLESS_BASE_URL || 'https://paperless.paccoco.com').replace(/\\/$/, '') + '/api/documents/post_document/'}}",
|
||||
"sendHeaders": true,
|
||||
"headerParameters": {
|
||||
"parameters": [
|
||||
{
|
||||
"name": "Authorization",
|
||||
"value": "={{'Token ' + $env.PAPERLESS_API_TOKEN}}"
|
||||
}
|
||||
]
|
||||
},
|
||||
"sendBody": true,
|
||||
"contentType": "multipart-form-data",
|
||||
"bodyParameters": {
|
||||
"parameters": [
|
||||
{
|
||||
"name": "document",
|
||||
"value": "data",
|
||||
"parameterType": "formBinaryData"
|
||||
},
|
||||
{
|
||||
"name": "title",
|
||||
"value": "={{$json.paperless_title}}"
|
||||
}
|
||||
]
|
||||
},
|
||||
"options": {
|
||||
"timeout": 120000
|
||||
}
|
||||
},
|
||||
"id": "paperless-upload",
|
||||
"name": "Upload to Paperless",
|
||||
"type": "n8n-nodes-base.httpRequest",
|
||||
"typeVersion": 4.2,
|
||||
"position": [
|
||||
-400,
|
||||
0
|
||||
]
|
||||
},
|
||||
{
|
||||
"parameters": {
|
||||
"jsCode": "\nconst intake = $('Normalize Intake Request').first().json;\nconst response = $json;\nconst taskId = response.task_id || response.id || response.document?.id || null;\nlet paperlessDocumentId = response.document_id || response.document?.id || null;\nif (!paperlessDocumentId && typeof response === 'object' && response?.id && !response?.task_id) {\n paperlessDocumentId = response.id;\n}\nreturn [{ json: {\n ...intake,\n paperless_task_id: taskId ? String(taskId) : null,\n paperless_document_id: paperlessDocumentId ? String(paperlessDocumentId) : null,\n paperless_response: response,\n uploaded_at: new Date().toISOString()\n}}];"
|
||||
},
|
||||
"id": "capture-paperless-response",
|
||||
"name": "Capture Paperless Response",
|
||||
"type": "n8n-nodes-base.code",
|
||||
"typeVersion": 2,
|
||||
"position": [
|
||||
-140,
|
||||
0
|
||||
]
|
||||
},
|
||||
{
|
||||
"parameters": {
|
||||
"operation": "executeQuery",
|
||||
"query": "UPDATE school_paperless_intake\nSET\n paperless_task_id = {{ $json.paperless_task_id ? \"'\" + $json.paperless_task_id.replace(/'/g, \"''\") + \"'\" : 'NULL' }},\n paperless_document_id = {{ $json.paperless_document_id ? \"'\" + $json.paperless_document_id.replace(/'/g, \"''\") + \"'\" : 'NULL' }},\n paperless_title = '{{ $json.paperless_title.replace(/'/g, \"''\") }}',\n status = 'uploaded',\n uploaded_at = '{{ $json.uploaded_at }}'::timestamptz,\n updated_at = NOW()\nWHERE intake_id = '{{ $json.intake_id }}';"
|
||||
},
|
||||
"id": "pg-mark-uploaded",
|
||||
"name": "Postgres Mark Uploaded",
|
||||
"type": "n8n-nodes-base.postgres",
|
||||
"typeVersion": 2.6,
|
||||
"position": [
|
||||
120,
|
||||
0
|
||||
],
|
||||
"credentials": {
|
||||
"postgres": {
|
||||
"id": "SETUP_REQUIRED",
|
||||
"name": "Shared Postgres"
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"parameters": {
|
||||
"jsCode": "\nreturn [{\n json: {\n ok: true,\n intake_id: $json.intake_id,\n status: 'uploaded',\n paperless_task_id: $json.paperless_task_id,\n paperless_document_id: $json.paperless_document_id,\n paperless_title: $json.paperless_title,\n stored_filename: $json.stored_filename,\n received_at: $json.received_at,\n uploaded_at: $json.uploaded_at\n }\n}];"
|
||||
},
|
||||
"id": "success-response",
|
||||
"name": "Return Success Payload",
|
||||
"type": "n8n-nodes-base.code",
|
||||
"typeVersion": 2,
|
||||
"position": [
|
||||
360,
|
||||
0
|
||||
]
|
||||
}
|
||||
],
|
||||
"connections": {
|
||||
"School Intake Webhook": {
|
||||
"main": [
|
||||
[
|
||||
{
|
||||
"node": "Normalize Intake Request",
|
||||
"type": "main",
|
||||
"index": 0
|
||||
}
|
||||
]
|
||||
]
|
||||
},
|
||||
"Normalize Intake Request": {
|
||||
"main": [
|
||||
[
|
||||
{
|
||||
"node": "Postgres Upsert Intake Received",
|
||||
"type": "main",
|
||||
"index": 0
|
||||
}
|
||||
]
|
||||
]
|
||||
},
|
||||
"Postgres Upsert Intake Received": {
|
||||
"main": [
|
||||
[
|
||||
{
|
||||
"node": "Upload to Paperless",
|
||||
"type": "main",
|
||||
"index": 0
|
||||
}
|
||||
]
|
||||
]
|
||||
},
|
||||
"Upload to Paperless": {
|
||||
"main": [
|
||||
[
|
||||
{
|
||||
"node": "Capture Paperless Response",
|
||||
"type": "main",
|
||||
"index": 0
|
||||
}
|
||||
]
|
||||
]
|
||||
},
|
||||
"Capture Paperless Response": {
|
||||
"main": [
|
||||
[
|
||||
{
|
||||
"node": "Postgres Mark Uploaded",
|
||||
"type": "main",
|
||||
"index": 0
|
||||
}
|
||||
]
|
||||
]
|
||||
},
|
||||
"Postgres Mark Uploaded": {
|
||||
"main": [
|
||||
[
|
||||
{
|
||||
"node": "Return Success Payload",
|
||||
"type": "main",
|
||||
"index": 0
|
||||
}
|
||||
]
|
||||
]
|
||||
}
|
||||
},
|
||||
"settings": {
|
||||
"executionOrder": "v1",
|
||||
"binaryMode": "separate"
|
||||
},
|
||||
"staticData": null,
|
||||
"meta": {
|
||||
"templateCredsSetupCompleted": false
|
||||
},
|
||||
"pinData": {},
|
||||
"tags": [
|
||||
{
|
||||
"name": "school"
|
||||
},
|
||||
{
|
||||
"name": "paperless"
|
||||
},
|
||||
{
|
||||
"name": "postgres"
|
||||
},
|
||||
{
|
||||
"name": "telegram"
|
||||
}
|
||||
]
|
||||
}
|
||||
294
n8n-workflows/19-paperless-school-metadata-enrichment.json
Normal file
294
n8n-workflows/19-paperless-school-metadata-enrichment.json
Normal file
@@ -0,0 +1,294 @@
|
||||
{
|
||||
"name": "Paperless School Intake Metadata Enrichment",
|
||||
"active": false,
|
||||
"isArchived": false,
|
||||
"nodes": [
|
||||
{
|
||||
"parameters": {
|
||||
"httpMethod": "POST",
|
||||
"path": "school/intake/paperless-enrich",
|
||||
"responseMode": "lastNode",
|
||||
"responseData": "allEntries"
|
||||
},
|
||||
"id": "paperless-school-webhook",
|
||||
"name": "Paperless School Webhook",
|
||||
"type": "n8n-nodes-base.webhook",
|
||||
"typeVersion": 2,
|
||||
"position": [
|
||||
-1160,
|
||||
0
|
||||
]
|
||||
},
|
||||
{
|
||||
"parameters": {
|
||||
"jsCode": "\nconst input = $input.first().json || {};\nconst body = input.body || input;\nconst candidates = [\n body.document_id,\n body.id,\n body.doc_id,\n body.pk,\n body.document_id?.id,\n body.document?.id,\n body.document\n].filter(v => v !== undefined && v !== null);\nlet raw = candidates[0];\nif (typeof raw === 'object' && raw !== null) raw = raw.id ?? raw.document_id ?? raw.pk ?? null;\nif (typeof raw === 'string' && /\\{.+\\}/.test(raw)) {\n throw new Error('Received unresolved template string instead of document id.');\n}\nconst documentId = Number(raw);\nif (!Number.isFinite(documentId) || documentId <= 0) {\n throw new Error('Could not resolve Paperless document id from webhook payload.');\n}\nreturn [{ json: { document_id: documentId } }];"
|
||||
},
|
||||
"id": "normalize-paperless-webhook",
|
||||
"name": "Normalize Paperless Webhook",
|
||||
"type": "n8n-nodes-base.code",
|
||||
"typeVersion": 2,
|
||||
"position": [
|
||||
-900,
|
||||
0
|
||||
]
|
||||
},
|
||||
{
|
||||
"parameters": {
|
||||
"method": "GET",
|
||||
"url": "={{($env.PAPERLESS_BASE_URL || 'https://paperless.paccoco.com').replace(/\\/$/, '') + '/api/documents/' + $json.document_id + '/'}}",
|
||||
"sendHeaders": true,
|
||||
"headerParameters": {
|
||||
"parameters": [
|
||||
{
|
||||
"name": "Authorization",
|
||||
"value": "={{'Token ' + $env.PAPERLESS_API_TOKEN}}"
|
||||
},
|
||||
{
|
||||
"name": "Accept",
|
||||
"value": "application/json"
|
||||
}
|
||||
]
|
||||
},
|
||||
"options": {
|
||||
"timeout": 120000
|
||||
}
|
||||
},
|
||||
"id": "fetch-paperless-document",
|
||||
"name": "Fetch Paperless Document",
|
||||
"type": "n8n-nodes-base.httpRequest",
|
||||
"typeVersion": 4.2,
|
||||
"position": [
|
||||
-640,
|
||||
0
|
||||
]
|
||||
},
|
||||
{
|
||||
"parameters": {
|
||||
"jsCode": "\nconst doc = $input.first().json || {};\nconst originalFilename = doc.original_file_name || doc.original_filename || '';\nconst baseName = String(originalFilename).replace(/\\.[^.]+$/, '');\nif (!baseName || !baseName.startsWith('school-')) {\n throw new Error('Document original filename does not contain expected intake id prefix: ' + originalFilename);\n}\nreturn [{ json: {\n intake_id: baseName,\n document_id: doc.id,\n current_title: doc.title || '',\n current_tags: Array.isArray(doc.tags) ? doc.tags : [],\n original_filename: originalFilename\n} }];"
|
||||
},
|
||||
"id": "extract-intake-id",
|
||||
"name": "Extract Intake ID",
|
||||
"type": "n8n-nodes-base.code",
|
||||
"typeVersion": 2,
|
||||
"position": [
|
||||
-380,
|
||||
0
|
||||
]
|
||||
},
|
||||
{
|
||||
"parameters": {
|
||||
"operation": "executeQuery",
|
||||
"query": "SELECT intake_id, class_name, assignment_name, submission_kind, semester, course_code, paper_kind, notes, source, telegram_chat_id, telegram_message_id, original_filename, stored_filename, checksum_sha256, status, paperless_task_id, paperless_document_id, paperless_title, received_at, uploaded_at, enriched_at FROM school_paperless_intake WHERE intake_id = '{{ $json.intake_id.replace(/'/g, \"''\") }}' LIMIT 1;"
|
||||
},
|
||||
"id": "pg-fetch-intake-record",
|
||||
"name": "Postgres Fetch Intake Record",
|
||||
"type": "n8n-nodes-base.postgres",
|
||||
"typeVersion": 2.6,
|
||||
"position": [
|
||||
-120,
|
||||
0
|
||||
],
|
||||
"credentials": {
|
||||
"postgres": {
|
||||
"id": "SETUP_REQUIRED",
|
||||
"name": "Shared Postgres"
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"parameters": {
|
||||
"jsCode": "\nconst intake = $input.first().json || {};\nif (!intake.intake_id) throw new Error('No intake record found in Postgres.');\nconst doc = $('Fetch Paperless Document').first().json || {};\nconst classTagMap = JSON.parse($env.PAPERLESS_TAG_CLASS_MAP_JSON || '{}');\nconst kindTagMap = JSON.parse($env.PAPERLESS_TAG_SUBMISSION_KIND_MAP_JSON || '{}');\nconst paperKindDocTypeMap = JSON.parse($env.PAPERLESS_DOCUMENT_TYPE_PAPER_KIND_MAP_JSON || '{}');\nconst classCorrespondentMap = JSON.parse($env.PAPERLESS_CORRESPONDENT_CLASS_MAP_JSON || '{}');\nconst currentTags = Array.isArray(doc.tags) ? doc.tags.map(Number).filter(Number.isFinite) : [];\nconst merged = new Set(currentTags);\nfor (const id of [$env.PAPERLESS_TAG_SCHOOL, $env.PAPERLESS_TAG_ASSIGNMENTS, $env.PAPERLESS_TAG_TELEGRAM]) {\n const n = Number(id);\n if (Number.isFinite(n) && n > 0) merged.add(n);\n}\nconst classTag = Number(classTagMap[intake.class_name]);\nif (Number.isFinite(classTag) && classTag > 0) merged.add(classTag);\nconst kindKey = String(intake.submission_kind || '').trim().toLowerCase();\nconst kindTag = Number(kindTagMap[kindKey]);\nif (Number.isFinite(kindTag) && kindTag > 0) merged.add(kindTag);\nconst title = [intake.class_name, intake.assignment_name, intake.submission_kind].filter(Boolean).join(' \u2014 ');\nconst payload = { title, tags: Array.from(merged) };\nconst documentType = Number(paperKindDocTypeMap[intake.paper_kind]);\nif (Number.isFinite(documentType) && documentType > 0) payload.document_type = documentType;\nconst correspondent = Number(classCorrespondentMap[intake.class_name]);\nif (Number.isFinite(correspondent) && correspondent > 0) payload.correspondent = correspondent;\nreturn [{ json: {\n intake_id: intake.intake_id,\n document_id: doc.id,\n payload,\n title,\n final_tag_ids: payload.tags,\n class_name: intake.class_name,\n assignment_name: intake.assignment_name,\n submission_kind: intake.submission_kind\n} }];"
|
||||
},
|
||||
"id": "build-paperless-update",
|
||||
"name": "Build Paperless Update",
|
||||
"type": "n8n-nodes-base.code",
|
||||
"typeVersion": 2,
|
||||
"position": [
|
||||
140,
|
||||
0
|
||||
]
|
||||
},
|
||||
{
|
||||
"parameters": {
|
||||
"method": "PATCH",
|
||||
"url": "={{($env.PAPERLESS_BASE_URL || 'https://paperless.paccoco.com').replace(/\\/$/, '') + '/api/documents/' + $json.document_id + '/'}}",
|
||||
"sendHeaders": true,
|
||||
"headerParameters": {
|
||||
"parameters": [
|
||||
{
|
||||
"name": "Authorization",
|
||||
"value": "={{'Token ' + $env.PAPERLESS_API_TOKEN}}"
|
||||
},
|
||||
{
|
||||
"name": "Content-Type",
|
||||
"value": "application/json"
|
||||
},
|
||||
{
|
||||
"name": "Accept",
|
||||
"value": "application/json"
|
||||
}
|
||||
]
|
||||
},
|
||||
"sendBody": true,
|
||||
"contentType": "json",
|
||||
"jsonBody": "={{$json.payload}}",
|
||||
"options": {
|
||||
"timeout": 120000
|
||||
}
|
||||
},
|
||||
"id": "update-paperless-document",
|
||||
"name": "Update Paperless Document",
|
||||
"type": "n8n-nodes-base.httpRequest",
|
||||
"typeVersion": 4.2,
|
||||
"position": [
|
||||
400,
|
||||
0
|
||||
]
|
||||
},
|
||||
{
|
||||
"parameters": {
|
||||
"operation": "executeQuery",
|
||||
"query": "UPDATE school_paperless_intake SET paperless_document_id = {{ Number($json.id || $('Build Paperless Update').first().json.document_id) }}, paperless_title = '{{ $('Build Paperless Update').first().json.title.replace(/'/g, \"''\") }}', status = 'enriched', enriched_at = NOW(), updated_at = NOW() WHERE intake_id = '{{ $('Build Paperless Update').first().json.intake_id.replace(/'/g, \"''\") }}';"
|
||||
},
|
||||
"id": "pg-mark-enriched",
|
||||
"name": "Postgres Mark Enriched",
|
||||
"type": "n8n-nodes-base.postgres",
|
||||
"typeVersion": 2.6,
|
||||
"position": [
|
||||
660,
|
||||
0
|
||||
],
|
||||
"credentials": {
|
||||
"postgres": {
|
||||
"id": "SETUP_REQUIRED",
|
||||
"name": "Shared Postgres"
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"parameters": {
|
||||
"jsCode": "\nconst update = $('Build Paperless Update').first().json;\nreturn [{ json: {\n ok: true,\n intake_id: update.intake_id,\n document_id: update.document_id,\n title: update.title,\n final_tag_ids: update.final_tag_ids,\n status: 'enriched'\n} }];"
|
||||
},
|
||||
"id": "return-enrichment-success",
|
||||
"name": "Return Enrichment Success",
|
||||
"type": "n8n-nodes-base.code",
|
||||
"typeVersion": 2,
|
||||
"position": [
|
||||
920,
|
||||
0
|
||||
]
|
||||
}
|
||||
],
|
||||
"connections": {
|
||||
"Paperless School Webhook": {
|
||||
"main": [
|
||||
[
|
||||
{
|
||||
"node": "Normalize Paperless Webhook",
|
||||
"type": "main",
|
||||
"index": 0
|
||||
}
|
||||
]
|
||||
]
|
||||
},
|
||||
"Normalize Paperless Webhook": {
|
||||
"main": [
|
||||
[
|
||||
{
|
||||
"node": "Fetch Paperless Document",
|
||||
"type": "main",
|
||||
"index": 0
|
||||
}
|
||||
]
|
||||
]
|
||||
},
|
||||
"Fetch Paperless Document": {
|
||||
"main": [
|
||||
[
|
||||
{
|
||||
"node": "Extract Intake ID",
|
||||
"type": "main",
|
||||
"index": 0
|
||||
}
|
||||
]
|
||||
]
|
||||
},
|
||||
"Extract Intake ID": {
|
||||
"main": [
|
||||
[
|
||||
{
|
||||
"node": "Postgres Fetch Intake Record",
|
||||
"type": "main",
|
||||
"index": 0
|
||||
}
|
||||
]
|
||||
]
|
||||
},
|
||||
"Postgres Fetch Intake Record": {
|
||||
"main": [
|
||||
[
|
||||
{
|
||||
"node": "Build Paperless Update",
|
||||
"type": "main",
|
||||
"index": 0
|
||||
}
|
||||
]
|
||||
]
|
||||
},
|
||||
"Build Paperless Update": {
|
||||
"main": [
|
||||
[
|
||||
{
|
||||
"node": "Update Paperless Document",
|
||||
"type": "main",
|
||||
"index": 0
|
||||
}
|
||||
]
|
||||
]
|
||||
},
|
||||
"Update Paperless Document": {
|
||||
"main": [
|
||||
[
|
||||
{
|
||||
"node": "Postgres Mark Enriched",
|
||||
"type": "main",
|
||||
"index": 0
|
||||
}
|
||||
]
|
||||
]
|
||||
},
|
||||
"Postgres Mark Enriched": {
|
||||
"main": [
|
||||
[
|
||||
{
|
||||
"node": "Return Enrichment Success",
|
||||
"type": "main",
|
||||
"index": 0
|
||||
}
|
||||
]
|
||||
]
|
||||
}
|
||||
},
|
||||
"settings": {
|
||||
"executionOrder": "v1"
|
||||
},
|
||||
"staticData": null,
|
||||
"meta": {
|
||||
"templateCredsSetupCompleted": false
|
||||
},
|
||||
"pinData": {},
|
||||
"tags": [
|
||||
{
|
||||
"name": "school"
|
||||
},
|
||||
{
|
||||
"name": "paperless"
|
||||
},
|
||||
{
|
||||
"name": "postgres"
|
||||
},
|
||||
{
|
||||
"name": "metadata"
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -15,6 +15,8 @@
|
||||
| 14 | Paperless Inbox Reminder | Scheduled (Monday 9 AM) | Queries Paperless for untagged documents; sends Gotify reminder with oldest items listed, or a ✅ clear if inbox is empty |
|
||||
| 15 | n8n Self-Health Monitor | Scheduled (every 15 min) | Checks n8n execution history for failures in the past hour; deduplicates via Postgres; pushes Gotify alert with workflow name and error summary |
|
||||
| 16 | NFS Mount Watchdog | Scheduled (every 5 min) | Probes NFS mount points on PD; if missing: runs mount script, re-probes, restarts affected containers (Plex, Audiobookshelf, Immich); notifies via Gotify on heal or escalation |
|
||||
| 18 | Telegram School Intake → Postgres → Paperless | `POST /webhook/school/intake/upload` | Accepts multipart schoolwork uploads plus class/assignment metadata, writes an intake record to shared Postgres, then uploads into Paperless with deterministic `intake_id` in the filename/title |
|
||||
| 19 | Paperless School Intake Metadata Enrichment | `POST /webhook/school/intake/paperless-enrich` | Handles Paperless post-consumption webhooks, looks up the intake record by deterministic filename, then applies deterministic title/tags and optional document-type/correspondent mapping |
|
||||
|
||||
## How to Import
|
||||
|
||||
@@ -67,9 +69,16 @@ Two steps required — both the `.env` file AND the `docker-compose.yaml` `envir
|
||||
|
||||
| Variable | Used by | How to get |
|
||||
|----------|---------|------------|
|
||||
| `PAPERLESS_API_TOKEN` | 04, 08 | Paperless → Settings → API token |
|
||||
| `PAPERLESS_API_TOKEN` | 04, 08, 18, 19 | Paperless → Settings → API token |
|
||||
| `PAPERLESS_TAG_TRANSCRIPTION` | 04 | Tag ID from Paperless API |
|
||||
| `PAPERLESS_TAG_LECTURE_NOTES` | 08 | Tag ID from Paperless API |
|
||||
| `PAPERLESS_TAG_SCHOOL` | 19 | Generic school tag ID from Paperless API |
|
||||
| `PAPERLESS_TAG_ASSIGNMENTS` | 19 | Generic assignments tag ID from Paperless API |
|
||||
| `PAPERLESS_TAG_TELEGRAM` | 19 | Tag ID for Telegram-submitted items |
|
||||
| `PAPERLESS_TAG_CLASS_MAP_JSON` | 19 | JSON object mapping class name → Paperless tag ID |
|
||||
| `PAPERLESS_TAG_SUBMISSION_KIND_MAP_JSON` | 19 | JSON object mapping submission kind → Paperless tag ID |
|
||||
| `PAPERLESS_DOCUMENT_TYPE_PAPER_KIND_MAP_JSON` | 19 | JSON object mapping paper kind → Paperless document type ID |
|
||||
| `PAPERLESS_CORRESPONDENT_CLASS_MAP_JSON` | 19 | JSON object mapping class name → Paperless correspondent ID |
|
||||
| `N8N_API_KEY` | 15 | n8n → Settings → API → Create API Key |
|
||||
| `LITELLM_API_KEY` | 01, 04, 08 | LiteLLM virtual key (already in .env) |
|
||||
|
||||
@@ -81,6 +90,13 @@ Two steps required — both the `.env` file AND the `docker-compose.yaml` `envir
|
||||
PAPERLESS_API_TOKEN: ${PAPERLESS_API_TOKEN}
|
||||
PAPERLESS_TAG_TRANSCRIPTION: ${PAPERLESS_TAG_TRANSCRIPTION}
|
||||
PAPERLESS_TAG_LECTURE_NOTES: ${PAPERLESS_TAG_LECTURE_NOTES}
|
||||
PAPERLESS_TAG_SCHOOL: ${PAPERLESS_TAG_SCHOOL}
|
||||
PAPERLESS_TAG_ASSIGNMENTS: ${PAPERLESS_TAG_ASSIGNMENTS}
|
||||
PAPERLESS_TAG_TELEGRAM: ${PAPERLESS_TAG_TELEGRAM}
|
||||
PAPERLESS_TAG_CLASS_MAP_JSON: ${PAPERLESS_TAG_CLASS_MAP_JSON}
|
||||
PAPERLESS_TAG_SUBMISSION_KIND_MAP_JSON: ${PAPERLESS_TAG_SUBMISSION_KIND_MAP_JSON}
|
||||
PAPERLESS_DOCUMENT_TYPE_PAPER_KIND_MAP_JSON: ${PAPERLESS_DOCUMENT_TYPE_PAPER_KIND_MAP_JSON}
|
||||
PAPERLESS_CORRESPONDENT_CLASS_MAP_JSON: ${PAPERLESS_CORRESPONDENT_CLASS_MAP_JSON}
|
||||
N8N_API_KEY: ${N8N_API_KEY}
|
||||
```
|
||||
|
||||
@@ -136,10 +152,11 @@ In Paperless-NGX, set up post-consumption webhooks to POST to:
|
||||
```
|
||||
https://n8n.paccoco.com/webhook/paperless/new-document
|
||||
https://n8n.paccoco.com/webhook/paperless/rag-ingest
|
||||
https://n8n.paccoco.com/webhook/school/intake/paperless-enrich
|
||||
```
|
||||
with body: `{"document_id": <id>}`
|
||||
|
||||
You can trigger both from the same Paperless event — workflow 03 classifies/tags the document, workflow 05 ingests it into RAG.
|
||||
You can trigger all three from the same Paperless event. Workflow 03 handles AI summary/tagging, workflow 05 handles RAG ingest, and workflow 19 re-applies deterministic school metadata for intake-managed documents.
|
||||
|
||||
### Grafana Webhook (already configured)
|
||||
|
||||
@@ -156,6 +173,19 @@ Works with Gitea, Forgejo, GitHub, and GitLab webhook payloads.
|
||||
|
||||
## Usage Examples
|
||||
|
||||
### Upload schoolwork from Telegram intake:
|
||||
```bash
|
||||
curl -X POST https://n8n.paccoco.com/webhook/school/intake/upload \
|
||||
-F "document=@essay-draft.pdf" \
|
||||
-F "class_name=English 101" \
|
||||
-F "assignment_name=Essay Draft 1" \
|
||||
-F "submission_kind=homework" \
|
||||
-F "semester=Fall 2026" \
|
||||
-F "course_code=ENG101" \
|
||||
-F "paper_kind=essay"
|
||||
```
|
||||
|
||||
|
||||
### Ingest a document into RAG:
|
||||
```bash
|
||||
curl -X POST https://n8n.paccoco.com/webhook/rag/ingest \
|
||||
@@ -259,6 +289,20 @@ curl -X POST https://n8n.paccoco.com/webhook/git/push \
|
||||
}'
|
||||
```
|
||||
|
||||
## Workflow 18 — Telegram School Intake → Postgres → Paperless
|
||||
|
||||
Import `18-school-paperless-intake.json` for a chat-driven schoolwork intake flow. Supporting notes live at `docs/operations/SCHOOL_PAPERLESS_INTAKE.md` and the tracked schema lives at `docs/reference/SCHOOL_INTAKE_POSTGRES_SCHEMA.sql`.
|
||||
|
||||
Highlights:
|
||||
- multipart upload endpoint: `POST /webhook/school/intake/upload`
|
||||
- required fields: `class_name`, `assignment_name`, `submission_kind`
|
||||
- optional fields: `semester`, `course_code`, `paper_kind`, `notes`, `telegram_chat_id`, `telegram_message_id`
|
||||
- writes intake metadata to shared Postgres before upload
|
||||
- sends deterministic `intake_id` in the Paperless filename and title correlation data
|
||||
|
||||
Testing fixture:
|
||||
- `fixtures/school-paperless-intake-webhook.curl.example.txt`
|
||||
|
||||
## Customizing RSS Feeds (Workflow 06)
|
||||
|
||||
Edit the "Define RSS Feeds" Code node to add/remove feeds. Default feeds:
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
# n8n Workflow Testing Status
|
||||
_Last updated: 2026-05-10_
|
||||
_Last updated: 2026-05-13_
|
||||
|
||||
---
|
||||
|
||||
@@ -105,3 +105,38 @@ _Last updated: 2026-05-10_
|
||||
- [ ] `PAPERLESS_TRIAGE_APPLY_SAFE=false` during initial validation
|
||||
|
||||
**Test approach:** Upload a harmless test document to Paperless, POST `{"document_id": <id>}` to the webhook, and confirm `doris-dashboard/data/paperless_review.json` contains a sanitized review item. Use a bill/due-date fixture to verify forced human review.
|
||||
|
||||
---
|
||||
|
||||
### 18 — Telegram School Intake → Postgres → Paperless v1.0
|
||||
**What it does:** Accepts multipart school-work upload plus metadata (`class_name`, `assignment_name`, `submission_kind`, optional semester/course_code/paper_kind), records it in shared Postgres, and uploads it into Paperless with a deterministic `intake_id`-based filename.
|
||||
|
||||
**Local artifact checks completed:**
|
||||
- [x] Workflow JSON validates with `python3 -m json.tool`
|
||||
- [x] Companion helper script validates with `node --check school/intake/submit_to_n8n.js`
|
||||
- [x] Class profile example JSON validates with `python3 -m json.tool`
|
||||
|
||||
**Pre-test checklist:**
|
||||
- [ ] Shared Postgres table `school_paperless_intake` created from `docs/reference/SCHOOL_INTAKE_POSTGRES_SCHEMA.sql`
|
||||
- [ ] `PAPERLESS_API_TOKEN` exposed to n8n through env vars
|
||||
- [ ] Postgres credential `Shared Postgres` created in n8n and wired into workflow 18
|
||||
- [ ] Webhook path `https://n8n.paccoco.com/webhook/school/intake/upload` reachable from Doris/OpenClaw helper
|
||||
|
||||
**Test approach:** POST a small DOCX/PDF through the intake webhook, confirm a row appears in `school_paperless_intake`, and verify the document lands in Paperless with filename prefix `school-...`.
|
||||
|
||||
---
|
||||
|
||||
### 19 — Paperless School Intake Metadata Enrichment v1.0
|
||||
**What it does:** Receives Paperless post-consumption webhook for intake-managed documents, resolves the deterministic `intake_id` from the stored filename, loads the intake record from Postgres, then reapplies deterministic title/tags and optional document type/correspondent mapping.
|
||||
|
||||
**Local artifact checks completed:**
|
||||
- [x] Workflow JSON validates with `python3 -m json.tool`
|
||||
|
||||
**Pre-test checklist:**
|
||||
- [ ] Paperless webhook configured to POST `https://n8n.paccoco.com/webhook/school/intake/paperless-enrich`
|
||||
- [ ] Workflow 18 already working so intake-managed filenames exist in Paperless
|
||||
- [ ] `PAPERLESS_API_TOKEN`, `PAPERLESS_TAG_SCHOOL`, `PAPERLESS_TAG_ASSIGNMENTS`, and `PAPERLESS_TAG_TELEGRAM` exposed to n8n
|
||||
- [ ] Optional mapping env vars supplied when ready: `PAPERLESS_TAG_CLASS_MAP_JSON`, `PAPERLESS_TAG_SUBMISSION_KIND_MAP_JSON`, `PAPERLESS_DOCUMENT_TYPE_PAPER_KIND_MAP_JSON`, `PAPERLESS_CORRESPONDENT_CLASS_MAP_JSON`
|
||||
- [ ] Postgres credential `Shared Postgres` created in n8n and wired into workflow 19
|
||||
|
||||
**Test approach:** After a workflow-18 upload is consumed by Paperless, POST `{"document_id": <id>}` to `/webhook/school/intake/paperless-enrich` and verify title/tags are updated and the DB row status becomes `enriched`.
|
||||
|
||||
@@ -0,0 +1,12 @@
|
||||
curl -X POST https://n8n.paccoco.com/webhook/school/intake/upload \
|
||||
-F "document=@essay-draft.pdf" \
|
||||
-F "class_name=English 101" \
|
||||
-F "assignment_name=Essay Draft 1" \
|
||||
-F "submission_kind=homework" \
|
||||
-F "semester=Fall 2026" \
|
||||
-F "course_code=ENG101" \
|
||||
-F "paper_kind=essay" \
|
||||
-F "notes=Submitted from Telegram chat intake" \
|
||||
-F "source=telegram" \
|
||||
-F "telegram_chat_id=123456789" \
|
||||
-F "telegram_message_id=987654321"
|
||||
Reference in New Issue
Block a user