83 lines
2.7 KiB
Markdown
83 lines
2.7 KiB
Markdown
# School Work Intake Pipeline (Telegram → Postgres → Paperless → n8n)
|
|
|
|
## Goal
|
|
|
|
Give Doris a reliable way to ingest school work from Telegram while preserving explicit metadata:
|
|
- class
|
|
- assignment
|
|
- submission kind (`draft`, `final`, etc.)
|
|
- optional semester / instructor / due date / paper kind
|
|
|
|
The design must be expandable for new classes and future non-paper workflows.
|
|
|
|
## Recommended architecture
|
|
|
|
1. **Telegram chat-driven intake**
|
|
- John sends file(s) to Doris on Telegram.
|
|
- Doris asks follow-up questions until required metadata is complete.
|
|
|
|
2. **n8n intake webhook**
|
|
- Doris submits the file + metadata to an intake webhook.
|
|
- The webhook creates a durable intake record in shared Postgres.
|
|
- The webhook stages/uploads the file into Paperless with a deterministic `intake_id` embedded in the filename.
|
|
|
|
3. **Paperless post-consumption webhook**
|
|
- After Paperless finishes processing the file, a second workflow fetches the Paperless document.
|
|
- That workflow extracts `intake_id`, looks up the Postgres intake record, and applies metadata.
|
|
|
|
4. **Metadata + AI enrichment**
|
|
- Deterministic metadata first: class tags, version tags, title rules, optional correspondent/document type IDs.
|
|
- AI second: document summary/note and any low-risk enrichment.
|
|
- Ambiguous cases can be routed to a review queue.
|
|
|
|
## Why this is better than folder-based consume ingestion
|
|
|
|
Folder paths are a decent hint, but they are not durable enough to be the only source of truth.
|
|
|
|
Problems with folder-only inference:
|
|
- folder context can disappear after import
|
|
- mixed uploads are easy to mislabel
|
|
- retries/reprocessing can grab the wrong document
|
|
- future use-cases (draft review, assignment history, instructor-specific rules) need explicit metadata anyway
|
|
|
|
## Shared DB requirements
|
|
|
|
Use shared Postgres, not SQLite.
|
|
|
|
Schema artifact:
|
|
- `docs/reference/SCHOOL_INTAKE_POSTGRES_SCHEMA.sql`
|
|
|
|
Core tables:
|
|
- `school_paperless_intake`
|
|
- `school_paperless_intake_events`
|
|
|
|
## Config-driven class profiles
|
|
|
|
Class/course behavior should live in config, not hardcoded workflow branches.
|
|
|
|
Suggested profile fields:
|
|
- `class_key`
|
|
- `display_name`
|
|
- `course_code`
|
|
- `default_tags`
|
|
- Paperless IDs for correspondent/document type/storage path when needed
|
|
- title template override
|
|
|
|
## Initial rollout target
|
|
|
|
- ENGL-1010
|
|
- HIST-2020
|
|
|
|
## OpenClaw helper artifact
|
|
|
|
Local helper:
|
|
- `school/intake/submit_to_n8n.js`
|
|
|
|
This gives Doris a simple handoff point after the Telegram conversation is complete.
|
|
|
|
## Live work still needed
|
|
|
|
- configure class profiles with real Paperless IDs / tag mappings
|
|
- decide Telegram delivery wording/confirmation behavior
|
|
- after any live `.env` changes for this pipeline, sync them into the encrypted homelab secrets repo (`docs/operations/SECRETS_MANAGEMENT.md`)
|