Files
truenas-stacks/docs/operations/SCHOOL_PAPERLESS_INTAKE.md
2026-05-16 15:54:20 +00:00

5.7 KiB

School Paperless Intake Workflow

Current workflow versions:

  • n8n-workflows/18-school-paperless-intake-v1.2.json
  • n8n-workflows/19-paperless-school-metadata-enrichment-v1.1.json

This workflow set is the Telegram chat-driven upload pipeline that sends schoolwork into Paperless-NGX with deterministic intake IDs and follow-up metadata enrichment.

Versioning rule

When these workflow artifacts change, the version must change too:

  • the filename keeps a -vX.Y suffix
  • the internal n8n workflow name also carries the same version
  • docs and testing notes should reference the current version explicitly

What it does

Workflow 18 — v1.2

  1. Accepts a multipart upload at POST /webhook/school/intake/upload
  2. Requires form fields:
    • class_name
    • assignment_name
    • submission_kind
  3. Accepts optional fields:
    • semester
    • course_code
    • paper_kind
    • notes
    • source (defaults to telegram)
    • telegram_chat_id
    • telegram_message_id
  4. Generates a deterministic intake_id from date + class + assignment + file checksum
  5. Builds a deterministic intake envelope from the request metadata and upload details
  6. Uploads the file to Paperless using the deterministic intake ID in the filename
  7. Marks the row as uploaded after Paperless accepts the upload

Current correlation model

  • Earlier drafts stored intake rows in shared Postgres.
  • The current live-tested v1.2 + v1.1 chain does not require Postgres in the webhook path.
  • Deterministic filename/title are the current source of correlation between upload and enrichment.
  • docs/reference/SCHOOL_INTAKE_POSTGRES_SCHEMA.sql is retained for a future DB-backed intake ledger if that gets reinstated.

Required environment variables

Add to automation/.env:

PAPERLESS_BASE_URL=https://paperless.paccoco.com
PAPERLESS_API_TOKEN=CHANGE_ME
LITELLM_API_KEY=CHANGE_ME
PAPERLESS_CORRESPONDENT_SCHOOL=
PAPERLESS_DOCUMENT_TYPE_SCHOOL=
PAPERLESS_TAG_SCHOOL=
PAPERLESS_TAG_ASSIGNMENTS=
PAPERLESS_TAG_TELEGRAM=
PAPERLESS_TAG_CLASS_MAP_JSON={}
PAPERLESS_TAG_SUBMISSION_KIND_MAP_JSON={}
PAPERLESS_DOCUMENT_TYPE_PAPER_KIND_MAP_JSON={}
PAPERLESS_CORRESPONDENT_CLASS_MAP_JSON={}
NODE_FUNCTION_ALLOW_BUILTIN=crypto

Add to the n8n service environment: block in automation/docker-compose.yaml:

      PAPERLESS_BASE_URL: ${PAPERLESS_BASE_URL}
      PAPERLESS_API_TOKEN: ${PAPERLESS_API_TOKEN}
      PAPERLESS_CORRESPONDENT_SCHOOL: ${PAPERLESS_CORRESPONDENT_SCHOOL}
      PAPERLESS_DOCUMENT_TYPE_SCHOOL: ${PAPERLESS_DOCUMENT_TYPE_SCHOOL}
      PAPERLESS_TAG_SCHOOL: ${PAPERLESS_TAG_SCHOOL}
      PAPERLESS_TAG_ASSIGNMENTS: ${PAPERLESS_TAG_ASSIGNMENTS}
      PAPERLESS_TAG_TELEGRAM: ${PAPERLESS_TAG_TELEGRAM}
      PAPERLESS_TAG_CLASS_MAP_JSON: ${PAPERLESS_TAG_CLASS_MAP_JSON}
      PAPERLESS_TAG_SUBMISSION_KIND_MAP_JSON: ${PAPERLESS_TAG_SUBMISSION_KIND_MAP_JSON}
      PAPERLESS_DOCUMENT_TYPE_PAPER_KIND_MAP_JSON: ${PAPERLESS_DOCUMENT_TYPE_PAPER_KIND_MAP_JSON}
      PAPERLESS_CORRESPONDENT_CLASS_MAP_JSON: ${PAPERLESS_CORRESPONDENT_CLASS_MAP_JSON}
      NODE_FUNCTION_ALLOW_BUILTIN: crypto

Notes:

  • PAPERLESS_CORRESPONDENT_SCHOOL and PAPERLESS_DOCUMENT_TYPE_SCHOOL should be numeric IDs if you want those fields auto-assigned.
  • PAPERLESS_TAG_CLASS_MAP_JSON, PAPERLESS_TAG_SUBMISSION_KIND_MAP_JSON, PAPERLESS_DOCUMENT_TYPE_PAPER_KIND_MAP_JSON, and PAPERLESS_CORRESPONDENT_CLASS_MAP_JSON are optional JSON maps used by workflow 19.
  • Leave optional Paperless IDs and JSON maps blank/default if you prefer to start with deterministic title + generic tags only.
  • Workflow 18 v1.1 no longer requires crypto, so NODE_FUNCTION_ALLOW_BUILTIN=crypto is optional instead of required.
  • Workflow 18 v1.2 fixes the Paperless multipart upload by using inputDataFieldName for the binary form field.
  • N8N_BLOCK_ENV_ACCESS_IN_NODE must remain false.

Paperless metadata behavior

The upload uses:

  • filename: <intake_id>.<ext>
  • title: <class_name> — <assignment_name> — <submission_kind>

Workflow 18 deliberately keeps the Paperless upload minimal and safe:

  • multipart document
  • deterministic title

The deterministic filename/title preserve the intake_id for later correlation.

Workflow 19 — v1.1

Workflow 19 handles the follow-up metadata pass after Paperless finishes processing:

  • fetch Paperless document by webhook document id
  • resolve intake_id from the stored filename
  • recover class/assignment/submission metadata from the deterministic filename and title
  • re-apply deterministic title
  • merge generic school tags plus optional class/submission-kind mappings
  • optionally set document type / correspondent from JSON env mappings

Testing

Use the example curl request in:

  • n8n-workflows/fixtures/school-paperless-intake-webhook.curl.example.txt

Minimum test checklist:

  1. Import the workflow into n8n.
  2. Import workflow 19 alongside workflow 18.
  3. Send the sample multipart request.
  4. Verify the uploaded Paperless document title and stored filename both include the deterministic intake ID.
  5. Trigger workflow 19 with the Paperless document id and verify title/tags enrichment succeeds.
  6. Re-submit the same file and metadata to confirm the same deterministic intake_id pattern is reused.

Known assumptions / caveats

  • Workflow 18 v1.2 assignment intake was live smoke-tested successfully on 2026-05-15.
  • Workflow 19 v1.1 is designed around the deterministic filename/title model rather than a Postgres lookup.
  • Workflow 18 v1.0 needed require('crypto'); workflow 18 v1.1 removed that dependency for easier live rollout; workflow 18 v1.2 replaced it with an internal stable hash.
  • If Paperless needs tags as repeated tags[] fields instead of a comma-separated tags field, adjust the HTTP Request node after import.