truenas-stacks/docs/operations/SCHOOL_PAPERLESS_INTAKE.md

# School Paperless Intake Workflow

Current workflow versions:
- `n8n-workflows/18-school-paperless-intake-v1.2.json`
- `n8n-workflows/19-paperless-school-metadata-enrichment-v1.1.json`

This workflow set is the Telegram chat-driven upload pipeline that sends schoolwork into Paperless-NGX with deterministic intake IDs and follow-up metadata enrichment.

## Versioning rule

When these workflow artifacts change, the version must change too:
- the filename keeps a `-vX.Y` suffix
- the internal n8n workflow `name` also carries the same version
- docs and testing notes should reference the current version explicitly

## What it does

### Workflow 18 — v1.2

1. Accepts a multipart upload at `POST /webhook/school/intake/upload`
2. Requires form fields:
   - `class_name`
   - `assignment_name`
   - `submission_kind`
3. Accepts optional fields:
   - `semester`
   - `course_code`
   - `paper_kind`
   - `notes`
   - `source` (defaults to `telegram`)
   - `telegram_chat_id`
   - `telegram_message_id`
4. Generates a deterministic `intake_id` from date + class + assignment + file checksum
5. Builds a deterministic intake envelope from the request metadata and upload details
6. Uploads the file to Paperless using the deterministic intake ID in the filename
7. Marks the row as `uploaded` after Paperless accepts the upload

## Current correlation model

- Earlier drafts stored intake rows in shared Postgres.
- The current live-tested `v1.2` + `v1.1` chain does **not** require Postgres in the webhook path.
- Deterministic filename/title are the current source of correlation between upload and enrichment.
- `docs/reference/SCHOOL_INTAKE_POSTGRES_SCHEMA.sql` is retained for a future DB-backed intake ledger if that gets reinstated.

## Required environment variables

Add to `automation/.env`:

```env
PAPERLESS_BASE_URL=https://paperless.paccoco.com
PAPERLESS_API_TOKEN=CHANGE_ME
LITELLM_API_KEY=CHANGE_ME
PAPERLESS_CORRESPONDENT_SCHOOL=
PAPERLESS_DOCUMENT_TYPE_SCHOOL=
PAPERLESS_TAG_SCHOOL=
PAPERLESS_TAG_ASSIGNMENTS=
PAPERLESS_TAG_TELEGRAM=
PAPERLESS_TAG_CLASS_MAP_JSON={}
PAPERLESS_TAG_SUBMISSION_KIND_MAP_JSON={}
PAPERLESS_DOCUMENT_TYPE_PAPER_KIND_MAP_JSON={}
PAPERLESS_CORRESPONDENT_CLASS_MAP_JSON={}
NODE_FUNCTION_ALLOW_BUILTIN=crypto
```

Add to the `n8n` service `environment:` block in `automation/docker-compose.yaml`:

```yaml
      PAPERLESS_BASE_URL: ${PAPERLESS_BASE_URL}
      PAPERLESS_API_TOKEN: ${PAPERLESS_API_TOKEN}
      PAPERLESS_CORRESPONDENT_SCHOOL: ${PAPERLESS_CORRESPONDENT_SCHOOL}
      PAPERLESS_DOCUMENT_TYPE_SCHOOL: ${PAPERLESS_DOCUMENT_TYPE_SCHOOL}
      PAPERLESS_TAG_SCHOOL: ${PAPERLESS_TAG_SCHOOL}
      PAPERLESS_TAG_ASSIGNMENTS: ${PAPERLESS_TAG_ASSIGNMENTS}
      PAPERLESS_TAG_TELEGRAM: ${PAPERLESS_TAG_TELEGRAM}
      PAPERLESS_TAG_CLASS_MAP_JSON: ${PAPERLESS_TAG_CLASS_MAP_JSON}
      PAPERLESS_TAG_SUBMISSION_KIND_MAP_JSON: ${PAPERLESS_TAG_SUBMISSION_KIND_MAP_JSON}
      PAPERLESS_DOCUMENT_TYPE_PAPER_KIND_MAP_JSON: ${PAPERLESS_DOCUMENT_TYPE_PAPER_KIND_MAP_JSON}
      PAPERLESS_CORRESPONDENT_CLASS_MAP_JSON: ${PAPERLESS_CORRESPONDENT_CLASS_MAP_JSON}
      NODE_FUNCTION_ALLOW_BUILTIN: crypto
```

Notes:
- `PAPERLESS_CORRESPONDENT_SCHOOL` and `PAPERLESS_DOCUMENT_TYPE_SCHOOL` should be numeric IDs if you want those fields auto-assigned.
- `PAPERLESS_TAG_CLASS_MAP_JSON`, `PAPERLESS_TAG_SUBMISSION_KIND_MAP_JSON`, `PAPERLESS_DOCUMENT_TYPE_PAPER_KIND_MAP_JSON`, and `PAPERLESS_CORRESPONDENT_CLASS_MAP_JSON` are optional JSON maps used by workflow 19.
- Leave optional Paperless IDs and JSON maps blank/default if you prefer to start with deterministic title + generic tags only.
- Workflow 18 v1.1 no longer requires `crypto`, so `NODE_FUNCTION_ALLOW_BUILTIN=crypto` is optional instead of required.
- Workflow 18 v1.2 fixes the Paperless multipart upload by using `inputDataFieldName` for the binary form field.
- `N8N_BLOCK_ENV_ACCESS_IN_NODE` must remain `false`.

## Paperless metadata behavior

The upload uses:
- filename: `<intake_id>.<ext>`
- title: `<class_name> — <assignment_name> — <submission_kind>`

Workflow 18 deliberately keeps the Paperless upload minimal and safe:
- multipart `document`
- deterministic title

The deterministic filename/title preserve the `intake_id` for later correlation.

### Workflow 19 — v1.1

Workflow 19 handles the follow-up metadata pass after Paperless finishes processing:
- fetch Paperless document by webhook document id
- resolve `intake_id` from the stored filename
- recover class/assignment/submission metadata from the deterministic filename and title
- re-apply deterministic title
- merge generic school tags plus optional class/submission-kind mappings
- optionally set document type / correspondent from JSON env mappings

## Testing

Use the example curl request in:
- `n8n-workflows/fixtures/school-paperless-intake-webhook.curl.example.txt`

Minimum test checklist:
1. Import the workflow into n8n.
2. Import workflow 19 alongside workflow 18.
3. Send the sample multipart request.
4. Verify the uploaded Paperless document title and stored filename both include the deterministic intake ID.
5. Trigger workflow 19 with the Paperless document id and verify title/tags enrichment succeeds.
6. Re-submit the same file and metadata to confirm the same deterministic `intake_id` pattern is reused.

## Known assumptions / caveats

- Workflow 18 `v1.2` assignment intake was live smoke-tested successfully on 2026-05-15.
- Workflow 19 `v1.1` is designed around the deterministic filename/title model rather than a Postgres lookup.
- Workflow 18 v1.0 needed `require('crypto')`; workflow 18 v1.1 removed that dependency for easier live rollout; workflow 18 v1.2 replaced it with an internal stable hash.
- If Paperless needs tags as repeated `tags[]` fields instead of a comma-separated `tags` field, adjust the HTTP Request node after import.