truenas-stacks/docs/planning/SCHOOL_INTAKE_PIPELINE.md

School Work Intake Pipeline (Telegram → Postgres → Paperless → n8n)

Goal

Give Doris a reliable way to ingest school work from Telegram while preserving explicit metadata:

• class
• assignment
• submission kind (draft, final, etc.)
• optional semester / instructor / due date / paper kind

The design must be expandable for new classes and future non-paper workflows.

Recommended architecture

1. Telegram chat-driven intake

   • John sends file(s) to Doris on Telegram.
   • Doris asks follow-up questions until required metadata is complete.

2. n8n intake webhook

   • Doris submits the file + metadata to an intake webhook.
   • The webhook creates a durable intake record in shared Postgres.
   • The webhook stages/uploads the file into Paperless with a deterministic intake_id embedded in the filename.

3. Paperless post-consumption webhook

   • After Paperless finishes processing the file, a second workflow fetches the Paperless document.
   • That workflow extracts intake_id, looks up the Postgres intake record, and applies metadata.

4. Metadata + AI enrichment

   • Deterministic metadata first: class tags, version tags, title rules, optional correspondent/document type IDs.
   • AI second: document summary/note and any low-risk enrichment.
   • Ambiguous cases can be routed to a review queue.

Why this is better than folder-based consume ingestion

Folder paths are a decent hint, but they are not durable enough to be the only source of truth.

Problems with folder-only inference:

• folder context can disappear after import
• mixed uploads are easy to mislabel
• retries/reprocessing can grab the wrong document
• future use-cases (draft review, assignment history, instructor-specific rules) need explicit metadata anyway

Shared DB requirements

Use shared Postgres, not SQLite.

Schema artifact:

• docs/reference/SCHOOL_INTAKE_POSTGRES_SCHEMA.sql

Core tables:

• school_paperless_intake
• school_paperless_intake_events

Config-driven class profiles

Class/course behavior should live in config, not hardcoded workflow branches.

Suggested profile fields:

• class_key
• display_name
• course_code
• default_tags
• Paperless IDs for correspondent/document type/storage path when needed
• title template override

Initial rollout target

• ENGL-1010
• HIST-2020

OpenClaw helper artifact

Local helper:

• school/intake/submit_to_n8n.js

This gives Doris a simple handoff point after the Telegram conversation is complete.

Live work still needed

• configure class profiles with real Paperless IDs / tag mappings
• decide Telegram delivery wording/confirmation behavior
• after any live .env changes for this pipeline, sync them into the encrypted homelab secrets repo (docs/operations/SECRETS_MANAGEMENT.md)