feat(n8n): add school paperless intake pipeline
This commit is contained in:
85
docs/planning/SCHOOL_INTAKE_PIPELINE.md
Normal file
85
docs/planning/SCHOOL_INTAKE_PIPELINE.md
Normal file
@@ -0,0 +1,85 @@
|
||||
# School Work Intake Pipeline (Telegram → Postgres → Paperless → n8n)
|
||||
|
||||
## Goal
|
||||
|
||||
Give Doris a reliable way to ingest school work from Telegram while preserving explicit metadata:
|
||||
- class
|
||||
- assignment
|
||||
- submission kind (`draft`, `final`, etc.)
|
||||
- optional semester / instructor / due date / paper kind
|
||||
|
||||
The design must be expandable for new classes and future non-paper workflows.
|
||||
|
||||
## Recommended architecture
|
||||
|
||||
1. **Telegram chat-driven intake**
|
||||
- John sends file(s) to Doris on Telegram.
|
||||
- Doris asks follow-up questions until required metadata is complete.
|
||||
|
||||
2. **n8n intake webhook**
|
||||
- Doris submits the file + metadata to an intake webhook.
|
||||
- The webhook creates a durable intake record in shared Postgres.
|
||||
- The webhook stages/uploads the file into Paperless with a deterministic `intake_id` embedded in the filename.
|
||||
|
||||
3. **Paperless post-consumption webhook**
|
||||
- After Paperless finishes processing the file, a second workflow fetches the Paperless document.
|
||||
- That workflow extracts `intake_id`, looks up the Postgres intake record, and applies metadata.
|
||||
|
||||
4. **Metadata + AI enrichment**
|
||||
- Deterministic metadata first: class tags, version tags, title rules, optional correspondent/document type IDs.
|
||||
- AI second: document summary/note and any low-risk enrichment.
|
||||
- Ambiguous cases can be routed to a review queue.
|
||||
|
||||
## Why this is better than folder-based consume ingestion
|
||||
|
||||
Folder paths are a decent hint, but they are not durable enough to be the only source of truth.
|
||||
|
||||
Problems with folder-only inference:
|
||||
- folder context can disappear after import
|
||||
- mixed uploads are easy to mislabel
|
||||
- retries/reprocessing can grab the wrong document
|
||||
- future use-cases (draft review, assignment history, instructor-specific rules) need explicit metadata anyway
|
||||
|
||||
## Shared DB requirements
|
||||
|
||||
Use shared Postgres, not SQLite.
|
||||
|
||||
Schema artifact:
|
||||
- `docs/reference/SCHOOL_INTAKE_POSTGRES_SCHEMA.sql`
|
||||
|
||||
Core tables:
|
||||
- `school_paperless_intake`
|
||||
- `school_paperless_intake_events`
|
||||
|
||||
## Config-driven class profiles
|
||||
|
||||
Class/course behavior should live in config, not hardcoded workflow branches.
|
||||
|
||||
Suggested profile fields:
|
||||
- `class_key`
|
||||
- `display_name`
|
||||
- `course_code`
|
||||
- `default_tags`
|
||||
- Paperless IDs for correspondent/document type/storage path when needed
|
||||
- title template override
|
||||
|
||||
## Initial rollout target
|
||||
|
||||
- ENGL-1010
|
||||
- HIST-2020
|
||||
|
||||
## OpenClaw helper artifact
|
||||
|
||||
Local helper:
|
||||
- `school/intake/submit_to_n8n.js`
|
||||
|
||||
This gives Doris a simple handoff point after the Telegram conversation is complete.
|
||||
|
||||
## Live work still needed
|
||||
|
||||
- import/activate the new n8n intake workflow
|
||||
- import/activate the Paperless enrichment workflow
|
||||
- create shared Postgres database/user/schema
|
||||
- configure class profiles with real Paperless IDs
|
||||
- decide Telegram delivery wording/confirmation behavior
|
||||
- test with real ENGL + HIST documents
|
||||
Reference in New Issue
Block a user