Add Doris KitchenOwl recipe importer

docs/operations/KITCHENOWL_RECIPE_IMPORT.md

@@ -0,0 +1,137 @@
+# KitchenOwl Recipe Import (Doris-managed)
+
+## Why this exists
+
+KitchenOwl's built-in import/scrape flow is unreliable for the way John and Leanne actually send recipe links around.
+
+This repo-side helper gives Doris a safer path:
+
+1. try KitchenOwl's native `/recipe/scrape` endpoint for sites it understands
+2. if that fails, fetch the page directly
+3. extract schema.org `Recipe` JSON-LD
+4. normalize ingredients/tags/timings into KitchenOwl's create-recipe payload
+5. optionally create the recipe through the KitchenOwl API
+
+That means Doris can ingest recipe links from chat without depending on KitchenOwl's flaky native importer.
+
+## Script
+
+```bash
+automation/bin/kitchenowl_recipe_import.py
+```
+
+Default mode is **dry-run**. It prints the normalized KitchenOwl payload and does **not** create anything unless `--create` is passed.
+
+## Required env vars
+
+Put these in a live `.env` file or export them ad hoc. Do **not** commit live secrets.
+
+```env
+KITCHENOWL_BASE_URL=https://owl.paccoco.com
+KITCHENOWL_API_TOKEN=CHANGE_ME
+KITCHENOWL_HOUSEHOLD_ID=1
+```
+
+Notes:
+- `KITCHENOWL_API_TOKEN` should be a long-lived token for a user who can create recipes in the target household.
+- `KITCHENOWL_HOUSEHOLD_ID` for the current live instance is `1` (`Stafford's`) unless that changes later.
+
+## Usage
+
+### Dry-run a link
+
+```bash
+cd /home/fizzlepoof/repos/truenas-stacks
+export KITCHENOWL_BASE_URL=https://owl.paccoco.com
+export KITCHENOWL_API_TOKEN=...redacted...
+export KITCHENOWL_HOUSEHOLD_ID=1
+
+automation/bin/kitchenowl_recipe_import.py --pretty \
+  "https://example.com/my-recipe"
+```
+
+### Actually create the recipe
+
+```bash
+automation/bin/kitchenowl_recipe_import.py --pretty --create \
+  "https://example.com/my-recipe"
+```
+
+### Import several links at once
+
+```bash
+automation/bin/kitchenowl_recipe_import.py --create --pretty \
+  "https://example.com/recipe-1" \
+  "https://example.com/recipe-2"
+```
+
+## Behavior details
+
+### Strategy order
+
+By default the helper tries:
+
+1. `GET /api/household/<id>/recipe/scrape?url=...`
+2. if KitchenOwl returns `Unsupported website` or otherwise fails, Doris falls back to direct HTML fetch + JSON-LD parse
+
+If you want to skip the built-in scrape and go straight to Doris mode:
+
+```bash
+automation/bin/kitchenowl_recipe_import.py --skip-kitchenowl-scrape --pretty URL
+```
+
+### Duplicate protection
+
+Before creating a recipe, the helper loads existing household recipes and skips creation if it finds:
+- an exact matching `source` URL, or
+- an exact matching recipe `name`
+
+To force duplicates anyway:
+
+```bash
+automation/bin/kitchenowl_recipe_import.py --create --allow-duplicates URL
+```
+
+### Visibility
+
+KitchenOwl visibility enum:
+- `0` = private (default)
+- `1` = link
+- `2` = public
+
+Example:
+
+```bash
+automation/bin/kitchenowl_recipe_import.py --create --visibility 0 URL
+```
+
+## What parses well
+
+The fallback parser works best on recipe pages that expose proper schema.org `Recipe` JSON-LD, which many decent recipe sites do.
+
+It pulls:
+- name
+- description
+- instructions
+- ingredient list
+- total/prep/cook time
+- yield/servings
+- categories/cuisine/keywords → KitchenOwl tags
+
+## Known limitations
+
+- Some sites block scraping aggressively; Doris first tries normal fetch, then falls back to `curl` headers for better odds.
+- Ingredient normalization is heuristic, not magic. It aims for a sane KitchenOwl item name plus a description/amount, but some weird ingredients may still need manual cleanup.
+- Image upload/import is not wired yet in this helper.
+- The helper expects structured JSON-LD. If a site hides the recipe entirely in client-side blobs or anti-bot nonsense, manual copy/paste may still be needed.
+- `GET /api/settings` on the live KitchenOwl instance currently returns `500`, but that does not block recipe import.
+
+## Doris workflow expectation
+
+Preferred operator flow when John or Leanne sends recipe links:
+
+1. Doris runs a dry-run first when the site is unfamiliar or suspicious.
+2. If the normalized payload looks sane, Doris reruns with `--create`.
+3. If the page is too messy, Doris asks for the raw ingredients/steps and imports it manually.
+
+That keeps KitchenOwl as the recipe store while making Doris the reliable intake layer.

docs/planning/TODO.md

@@ -1,27 +1,27 @@
 # Homelab TODO
 
 ## Security (do ASAP)
-- [ ] Regenerate Wings token on N.O.M.A.D. (was exposed in chat)
-- [ ] Regenerate N.O.M.A.D. Newt secret (was exposed in chat)
+- [x] Regenerate Wings token on N.O.M.A.D. (was exposed in chat) — completed 2026-05-15
+- [x] Regenerate N.O.M.A.D. Newt secret (was exposed in chat) — completed 2026-05-15
 
 ## Infrastructure
 - [ ] Deploy Pi-hole 3-node HA cluster (PD + Serenity + RPi4) — see docs/planning/DEPLOY_PIHOLE.md
   - RPi4 IP still TBD — fill in before deploying
-- [ ] Deploy Kima-Hub — see docs/planning/DEPLOY_KIMA_HUB.md
+- [x] Deploy Kima-Hub — completed 2026-05-06; docs/planning/DEPLOY_KIMA_HUB.md now serves as deployment/repair reference
 - [ ] Set up off-site backup for vital data (appdata, databases, config repo, photos)
 - [ ] Remove dead Unraid-Cloudflared-Tunnel container from Serenity
 - [ ] Serenity malcolm pool capacity planning (heavily utilized)
-- [ ] Add Pangolin resource for `ai.paccoco.com` → OpenWebUI (port 8282)
-- [ ] Add Cloudflare DNS A record for `ai.paccoco.com`
+- [x] Add Pangolin resource for `openwebui.paccoco.com` → OpenWebUI (port 8282) — completed 2026-05-15
+- [x] Add Cloudflare DNS record for `openwebui.paccoco.com` — completed 2026-05-15
 - [ ] Install fresh editor on PD
-- [ ] Set up private Gitea repo for .env file backups
+- [x] Set up private Gitea repo for .env file backups — 2026-05-14
 
 ## AI Stack
-- [ ] Pull `qwen2.5:14b` on PD ollama: `sudo docker exec -it ollama ollama pull qwen2.5:14b`
-- [ ] Create OpenWebUI account and set system prompt from HOMELAB_AI_CONTEXT.md
-- [ ] Disable signups in OpenWebUI after creating account
-- [ ] Connect Rocinante Ollama to OpenWebUI (Admin → Settings → Connections)
-- [ ] Confirm `OLLAMA_HOST=0.0.0.0` set on Rocinante and Ollama service restarted
+- [x] Pull `qwen2.5:14b` on PD ollama — completed 2026-05-15
+- [x] Create OpenWebUI account and set system prompt from HOMELAB_AI_CONTEXT.md — completed 2026-05-15
+- [x] Disable signups in OpenWebUI after creating account — completed 2026-05-15
+- [x] Connect Rocinante Ollama to OpenWebUI (Admin → Settings → Connections) — completed 2026-05-15
+- [x] Confirm `OLLAMA_HOST=0.0.0.0` set on Rocinante and Ollama service restarted — completed 2026-05-15
 
 ## Healthchecks
 - [ ] Add healthcheck to ix-plex-plex-1
@@ -43,11 +43,12 @@
 - [ ] Add remaining env vars to n8n docker-compose + .env: `PAPERLESS_API_TOKEN`, `PAPERLESS_TAG_TRANSCRIPTION`, `PAPERLESS_TAG_LECTURE_NOTES`, `N8N_API_KEY` (see n8n-workflows/README.md for docker-compose.yaml changes required)
 - [ ] Set up `PD SSH` credential in n8n for workflow 16 (see n8n-workflows/README.md)
 - [ ] Add sudoers NOPASSWD rule for n8n watchdog (see n8n-workflows/README.md)
-- [ ] Import and activate upgraded workflows: 04, 08 (01 done — v1.4 active and tested)
-- [ ] Import and activate new workflows: 14, 15 (16 already active)
-- [ ] Configure class profiles + Paperless metadata mapping for current courses (ENGL-1010, HIST-2020, future classes)
+- [x] Import and activate upgraded workflows: 04, 08 (01 done — v1.4 active and tested) — completed 2026-05-15
+- [x] Import and activate new workflows: 14, 15 (16 already active) — completed 2026-05-15
+- [x] Configure class profiles + Paperless metadata mapping for current courses (ENGL-1010, HIST-2020, future classes) — completed 2026-05-15
 - [x] Delete legacy duplicate Paperless notes for school docs 42–49 after approval / with proper token context — completed 2026-05-14; docs now each have one canonical note
-- [ ] Set up DB and Docker backups to Serenity following homelab SOP
+- [x] Set up DB and Docker backups to Serenity following homelab SOP — completed 2026-05-15
+  - Live `.env`, SSH trust, scheduler deployment, first run, and verification completed on PD
 
 ## Completed
 - [x] Deploy Gotify (Phase 1) — 2026-05-05
@@ -67,3 +68,4 @@
 - [x] Workflow 01 fully tested end-to-end (dedup, LiteLLM, Gotify) — v1.4 active — 2026-05-09
 - [x] Add LITELLM_API_KEY + N8N_BLOCK_ENV_ACCESS_IN_NODE to n8n docker-compose — 2026-05-09
 - [x] Build Telegram chat-driven school-work intake for Paperless using shared Postgres, deterministic `intake_id`, and class/version metadata — 2026-05-14
+- [x] Build Doris-managed KitchenOwl recipe importer flow with native-scrape fallback, schema.org JSON-LD parsing, duplicate checks, and dry-run support — 2026-05-15