Compare commits
6 Commits
db8dce8c8b
...
main
| Author | SHA1 | Date | |
|---|---|---|---|
|
482a490d3d | ||
|
|
1eb4bb9981 | ||
|
|
43b94813ca | ||
|
|
e19d4a5be3 | ||
|
|
28b2edf2a3 | ||
|
|
7a67a2febc |
@@ -7,6 +7,7 @@ Phase 2 (AI stack) is fully deployed and verified on PlausibleDeniability (10.5.
|
||||
## What's Done
|
||||
|
||||
### Phase 2 — AI Stack (COMPLETE)
|
||||
Historical note: this handoff reflects the original PD-first AI deployment snapshot, not the current Honcho routing policy.
|
||||
All 6 containers running and verified on PD (10.5.30.6):
|
||||
- **Ollama:** port 11434 — qwen2.5:14b loaded
|
||||
- **OpenWebUI:** port 8282 (→8080)
|
||||
|
||||
@@ -575,7 +575,7 @@
|
||||
575|Trigger: Cron node (every Sunday at 10:00 AM)
|
||||
576|Step 1: Postgres node → SELECT watch history for past 7 days
|
||||
577|Step 2: Format data as structured text
|
||||
578|Step 3: HTTP Request → Ollama API (POST to PD's qwen2.5:14b)
|
||||
578|Step 3: HTTP Request → Ollama API (POST to the local/default automation model host, preferably N.O.M.A.D.-local rather than PD)
|
||||
579| - Prompt: "Summarize this week's viewing in a fun digest: {data}"
|
||||
580|Step 4: HTTP Request → Gotify (send digest)
|
||||
581|```
|
||||
@@ -1093,9 +1093,9 @@
|
||||
1388|
|
||||
1389|LiteLLM sits in front of your three Ollama instances and presents a single OpenAI-compatible API at `http://litellm:4000`. It handles:
|
||||
1390|
|
||||
1391|- **Model routing** — requests for `qwen3:32b` go to ROCINANTE, `qwen2.5:14b` goes to PD, `phi4` goes to N.O.M.A.D.
|
||||
1392|- **Failover** — if ROCINANTE is offline, LiteLLM can fall back to PD automatically
|
||||
1393|- **Load balancing** — distribute requests across instances running the same model
|
||||
- **Model routing** — current reliable default is N.O.M.A.D.-local Ollama for background/automation paths, with PD kept available for shared light-tier use and Rocinante treated as optional/manual heavy capacity.
|
||||
- **Failover** — avoid designs that require Rocinante to be online; degrade to N.O.M.A.D.-local or explicitly operator-invoked PD paths instead of automatic dependence on a personal PC.
|
||||
- **Load balancing** — distribute requests across instances running the same model when you have multiple dependable backends, but do not assume Rocinante qualifies as one.
|
||||
1394|- **Usage tracking** — logs token counts, latency, and costs per model/user via its built-in database
|
||||
1395|
|
||||
1396|### 7.2 — Scaffold Directories
|
||||
@@ -1621,17 +1621,23 @@
|
||||
1916|{
|
||||
1917| "models": [
|
||||
1918| {
|
||||
1919| "title": "PD - qwen2.5:14b",
|
||||
1919| "title": "N.O.M.A.D. - qwen2.5:3b",
|
||||
1920| "provider": "ollama",
|
||||
1921| "model": "qwen2.5:14b",
|
||||
1922| "apiBase": "http://PD_TAILSCALE_IP:11434"
|
||||
1921| "model": "qwen2.5:3b",
|
||||
1922| "apiBase": "http://NOMAD_LOCAL_OR_TAILSCALE_IP:11434"
|
||||
1923| },
|
||||
1924| {
|
||||
1925| "title": "ROCINANTE - qwen3:32b",
|
||||
1925| "title": "PD - shared light tier (manual / fallback)",
|
||||
1926| "provider": "ollama",
|
||||
1927| "model": "qwen3:32b",
|
||||
1928| "apiBase": "http://ROCINANTE_TAILSCALE_IP:11434"
|
||||
1929| }
|
||||
1927| "model": "qwen2.5:14b",
|
||||
1928| "apiBase": "http://PD_TAILSCALE_IP:11434"
|
||||
1929| },
|
||||
1930| {
|
||||
1931| "title": "ROCINANTE - opportunistic heavy",
|
||||
1932| "provider": "ollama",
|
||||
1933| "model": "qwen3:32b",
|
||||
1934| "apiBase": "http://ROCINANTE_TAILSCALE_IP:11434"
|
||||
1935| }
|
||||
1930| ],
|
||||
1931| "tabAutocompleteModel": {
|
||||
1932| "title": "N.O.M.A.D. - phi4",
|
||||
|
||||
@@ -21,7 +21,7 @@ home/ # kitchenowl, donetick, shlink, qui, dakboard bridges, doris-da
|
||||
dev/ # gitea, rackpeek
|
||||
ai/ # ollama, openwebui
|
||||
headscale/ # pilot self-hosted tailnet control plane + Headplane UI on PD
|
||||
meshtastic/ # meshmonitor, mqtt-proxy, tileserver, meshtastic-map
|
||||
meshtastic/ # meshmonitor, tileserver, meshtastic-map, and MeshCore/MeshMonitor integration notes
|
||||
automation/ # planned: gotify, n8n
|
||||
search/ # planned: qdrant
|
||||
docs/ # all homelab documentation
|
||||
|
||||
@@ -40,7 +40,8 @@ Current live homelab stack plus the planned direction for the upgraded PD platfo
|
||||
- **Dockhand** (3230) — Docker management UI
|
||||
|
||||
### AI Layer
|
||||
- **LiteLLM** (4000) — unified LLM proxy; routes to Ollama on PD (light) and Rocinante (heavy)
|
||||
- **LiteLLM** (4000) — unified LLM proxy; PD remains the shared light tier, while Rocinante is an opportunistic heavy tier when the PC is online
|
||||
- **Honcho** — memory/deriver stack on N.O.M.A.D. using local Ollama first so PD outages do not cascade into MeshMonitor/database incidents
|
||||
- **OpenWebUI** (8282) — chat UI with RAG, STT, and web search
|
||||
- **Qdrant** (6333/6334) — vector database for RAG
|
||||
- **Whisper** (8786 / 8787) — CPU speech-to-text on N.O.M.A.D. plus CUDA speech-to-text on Rocinante
|
||||
|
||||
@@ -2,7 +2,7 @@
|
||||
|
||||
All homelab services, their hosts, ports, and current status.
|
||||
|
||||
*Last updated: 2026-05-27*
|
||||
*Last updated: 2026-06-13*
|
||||
|
||||
## Shared Databases (PD)
|
||||
|
||||
@@ -40,8 +40,8 @@ All homelab services, their hosts, ports, and current status.
|
||||
| Whisper (faster-whisper, CPU) | N.O.M.A.D. (10.5.30.7) | 8786 | ✅ Active |
|
||||
| Whisper (faster-whisper, CUDA large-v3) | Rocinante (10.5.30.112) | 8787 | ⚠️ Re-check from N.O.M.A.D. if Whisper-specific traffic is needed; Ollama-only recovery was validated 2026-05-27 |
|
||||
| SearXNG | PD | 8888 | ✅ Active |
|
||||
| Ollama — light tier | PD (10.5.30.6) | 11434 | ✅ Active |
|
||||
| Ollama — heavy tier | Rocinante (10.5.30.112) | 11434 | ✅ Reachable from N.O.M.A.D.; re-validated 2026-05-27 |
|
||||
| Ollama — light tier | PD (10.5.30.6) | 11434 | ✅ Active; keep routine Honcho load off this host to protect MeshMonitor/shared Postgres |
|
||||
| Ollama — heavy tier | Rocinante (10.5.30.112) | 11434 | ⚠️ Personal PC / opportunistic only; do not depend on it for routine background routing |
|
||||
| Reranker (TEI) | Serenity (10.5.30.5) | 9787 | ✅ Active |
|
||||
|
||||
OpenClaw is no longer treated as an active PD service in the operator inventory. Only helper-artifact references remain under the school-intake planning docs.
|
||||
@@ -100,7 +100,7 @@ OpenClaw is no longer treated as an active PD service in the operator inventory.
|
||||
|
||||
| Service | Host | Port | Status |
|
||||
|---------|------|------|--------|
|
||||
| MeshCore to MQTT relay | N.O.M.A.D. (10.5.30.7) | outbound MQTT only | ✅ Running as Docker container `mctomqtt` (`mctomqtt:latest`); config bind-mounted from `/etc/mctomqtt` |
|
||||
| MeshCore companion observer | N.O.M.A.D. (10.5.30.7) | outbound MQTT only | ✅ Running as systemd service `meshcore-capture`; install root `/home/fizzlepoof/.meshcore-packet-capture`; direct serial connection to the Heltec companion node |
|
||||
|
||||
## Local file intake
|
||||
|
||||
|
||||
@@ -1,9 +1,14 @@
|
||||
# Meshtastic Stack
|
||||
|
||||
## Overview
|
||||
Self-hosted Meshtastic monitoring and management stack running on PlausibleDeniability.
|
||||
Self-hosted MeshMonitor stack running on PlausibleDeniability, plus the current operator note for the separate NOMAD-side MeshCore companion observer.
|
||||
|
||||
Related NOMAD-side bridge note: a separate MeshCore-to-MQTT relay is running on N.O.M.A.D. as Docker container `mctomqtt` (`mctomqtt:latest`), with config bind-mounted from `/etc/mctomqtt`. The package also includes a systemd unit under `/opt/mctomqtt`, but that is not the active runtime on this host.
|
||||
Related NOMAD-side bridge note: the legacy `mctomqtt` Docker relay was retired on 2026-06-13. N.O.M.A.D. now uses the LetsMesh companion observer flow via `meshcore-capture.service`, running from `/home/fizzlepoof/.meshcore-packet-capture` and connecting directly to the Heltec over `/dev/serial/by-id/...`.
|
||||
|
||||
Important split:
|
||||
- **PD** hosts the MeshMonitor web/UI stack.
|
||||
- **NOMAD** owns the USB-attached Heltec companion observer via `meshcore-capture.service`.
|
||||
- Do not assume MeshMonitor and the companion observer are the same runtime path.
|
||||
|
||||
## Stack Location
|
||||
`/mnt/docker-ssd/docker/compose/meshtastic/`
|
||||
@@ -12,29 +17,43 @@ Related NOMAD-side bridge note: a separate MeshCore-to-MQTT relay is running on
|
||||
|
||||
| Container | Image | Port | Purpose |
|
||||
|-----------|-------|------|---------|
|
||||
| meshmonitor | ghcr.io/yeraze/meshmonitor:4.2.0 | 8081→3001 | Main UI + backend |
|
||||
| meshmonitor (virtual node) | — | 4404 | TCP virtual node server |
|
||||
| meshmonitor-tileserver | maptiler/tileserver-gl-light | 8082→8080 | Offline map tiles |
|
||||
| meshmonitor-mqtt-proxy | ghcr.io/ln4cy/mqtt-proxy:master | — | MQTT→TCP bridge |
|
||||
| meshmonitor | ghcr.io/yeraze/meshmonitor:latest | 8081→3001, 4404→4404, 1883→1883 | Main UI + backend; live compose publishes the virtual-node port and embedded MQTT broker directly from this container, and now maps `/dev/ttyACM0` into the container for native MeshCore USB sources |
|
||||
| meshmonitor (virtual node) | — | 4404 | TCP virtual node server exposed directly by the meshmonitor container |
|
||||
| meshmonitor (embedded MQTT broker) | — | 1883 | Embedded Aedes-based MQTT broker exposed directly by the meshmonitor container |
|
||||
| meshmonitor-tileserver | maptiler/tileserver-gl-light:latest | 8082→8080 | Offline map tiles |
|
||||
| meshmonitor-upgrader | docker:latest | — | Auto-upgrade watchdog |
|
||||
|
||||
Live runtime verified on 2026-07-03:
|
||||
- `docker compose ps` on PD showed `meshmonitor`, `meshmonitor-tileserver`, and `meshmonitor-upgrader` running.
|
||||
- Repo compose and live runtime now agree that there is no separate `meshmonitor-mqtt-proxy` sidecar in the active stack.
|
||||
- The embedded MQTT broker source `Meshcore Peachtree Observer` is configured in MeshMonitor and logs `MQTT broker listening on 0.0.0.0:1883`.
|
||||
- A native MeshCore USB source `PT Meshcore` is now auto-connected in MeshMonitor via `/dev/ttyACM0` after the compose file was updated to map the device into the container.
|
||||
- NOMAD `meshcore-capture.service` was later returned to its three upstream brokers after the embedded-broker test path proved unnecessary for real MeshCore ingestion.
|
||||
|
||||
## Architecture
|
||||
```
|
||||
Meshtastic Node (10.5.1.172)
|
||||
↓ MQTT
|
||||
meshmonitor-mqtt-proxy
|
||||
↓ TCP (port 4404)
|
||||
meshmonitor virtual node server
|
||||
↓
|
||||
meshmonitor backend
|
||||
Trusted-LAN mesh node (10.5.1.120)
|
||||
↓ direct node connection configured by MESHTASTIC_NODE_IP
|
||||
meshmonitor backend on PD
|
||||
↓
|
||||
PostgreSQL (shared-postgres, db: meshmonitor)
|
||||
|
||||
Separate path:
|
||||
NOMAD Heltec companion observer (`meshcore-capture.service`)
|
||||
↓ upstream MQTT brokers (LetsMesh US / EU + NashMe)
|
||||
|
||||
PD native MeshCore USB source (`PT Meshcore`)
|
||||
↓ `/dev/ttyACM0` mapped into the meshmonitor container
|
||||
↓ meshcore.js native companion backend
|
||||
```
|
||||
|
||||
Note: the embedded MeshMonitor broker now accepts the NOMAD connection on `10.5.30.6:1883`, but current MeshMonitor logs still show protobuf decode warnings for MeshCore packet payloads. Treat this as confirmed broker fanout connectivity, not yet confirmed first-class MeshCore ingestion.
|
||||
|
||||
## Node Connection
|
||||
- Node IP: `10.5.1.172`
|
||||
- Connection type: MQTT via mqtt-proxy (not direct TCP)
|
||||
- mqtt-proxy connects to meshmonitor's virtual node server on port 4404
|
||||
- PD MeshMonitor compose is currently configured with `MESHTASTIC_NODE_IP=10.5.1.120`
|
||||
- MeshMonitor itself publishes port `4404` for the virtual-node path and `1883` for the embedded MQTT broker; there is no separate `meshmonitor-mqtt-proxy` service in the active stack
|
||||
- For native MeshCore USB sources on PD, the live compose now maps `/dev/ttyACM0` directly into the `meshmonitor` container
|
||||
- NOMAD's companion observer is a different ingest path and should not be documented as a MeshMonitor sidecar
|
||||
|
||||
## Storage
|
||||
- Appdata: `/mnt/tank/docker/appdata/meshmonitor/data` → `/data`
|
||||
@@ -60,7 +79,7 @@ mkdir -p /mnt/docker-ssd/docker/compose/meshtastic/scripts
|
||||
|
||||
## Known Warnings
|
||||
- Node `!dd972536` has a low-entropy public key — this is a device configuration issue, not a meshmonitor bug
|
||||
- Frequent connect/disconnect cycles in logs are normal when the MQTT proxy reconnects
|
||||
- MeshMonitor logs can still show noisy reconnect/admin-session churn during node or radio instability; do not confuse that with the retired NOMAD `mctomqtt` path
|
||||
|
||||
## Tiles
|
||||
Map tiles stored in `./tiles/`. Currently includes `zurich_switzerland.mbtiles` (34MB).
|
||||
@@ -68,5 +87,4 @@ Tiles directory is gitignored. To add new tiles, download `.mbtiles` files and p
|
||||
|
||||
## TODO
|
||||
- [ ] Document all running meshtastic-map container config
|
||||
- [ ] Document meshtastic-map-mosquitto and meshtastic-mqtt containers
|
||||
- [ ] Add healthcheck to meshmonitor container
|
||||
|
||||
@@ -226,43 +226,27 @@ These are expected to be managed by Wings/Pelican rather than a local compose fi
|
||||
- Sorter service: `minerva-localsend-autosort.service`
|
||||
- Sorter script: `/home/fizzlepoof/.local/bin/minerva-localsend-autosort.py`
|
||||
|
||||
### MeshCore to MQTT relay
|
||||
- Install date: 2026-05-18
|
||||
- Install root: `/opt/mctomqtt`
|
||||
- Config root: `/etc/mctomqtt`
|
||||
- Service account: `mctomqtt:mctomqtt`
|
||||
- Installer metadata: `.version_info` reports `installer_version: 1.3.0.0-preview`, repo `Cisien/meshcoretomqtt`, branch `main`, commit `10f7c0e`
|
||||
- Detected install type marker: `/opt/mctomqtt/.install_type` contains `docker`
|
||||
- Runtime model:
|
||||
- Active runtime is a standalone Docker container named `mctomqtt`
|
||||
- Image: `mctomqtt:latest`
|
||||
- Command: `python3 /opt/mctomqtt/mctomqtt.py`
|
||||
- User: `mctomqtt`
|
||||
- Restart policy: `unless-stopped`
|
||||
- Network mode: `bridge`
|
||||
- Config bind mount: `/etc/mctomqtt:/etc/mctomqtt:ro`
|
||||
- Device passthrough: the Heltec serial device is mapped directly into the container at the same `/dev/serial/by-id/... ` path
|
||||
- Systemd note:
|
||||
- The packaged repo also includes `/opt/mctomqtt/mctomqtt.service`, but it is not the active runtime on NOMAD
|
||||
- No installed unit was present in `/etc/systemd/system` or `/lib/systemd/system`
|
||||
- The bundled unit expects `/opt/mctomqtt/venv/bin/python3`, but no `venv/` directory exists under `/opt/mctomqtt`
|
||||
- Treat the systemd unit as an unused package artifact for this host unless the deployment model changes later
|
||||
- Config layering:
|
||||
- Base reference: `/etc/mctomqtt/config.toml`
|
||||
- Preset overlay: `/etc/mctomqtt/config.d/10-meshmapper.toml`
|
||||
- Local overrides: `/etc/mctomqtt/config.d/99-user.toml`
|
||||
- Current local overrides observed:
|
||||
- IATA region code set to `BNA`
|
||||
- Serial device pinned to `/dev/serial/by-id/usb-Espressif_Systems_heltec_wifi_lora_32_v4__16_MB_FLASH__2_MB_PSRAM__441BF670B684-if00`
|
||||
- Enabled broker targets include MeshMapper, LetsMesh US, and NashMe
|
||||
- Credentials/secrets are stored inline in `99-user.toml`; do not mirror them into git-backed docs
|
||||
- Container health observed during inspection:
|
||||
- Logs show the relay loading all three config layers, opening the Heltec serial device, and connecting successfully to MeshMapper, NashMe, and LetsMesh US after restart
|
||||
- The current container had been up about 5 minutes during inspection with `MQTT: 3/3`, zero failures, and live RX air-time reported
|
||||
- An earlier start logged one DNS resolution failure for a broker label shown as `custom-2`, but the current running container recovered and all three named brokers connected cleanly
|
||||
- Container logs / inspection:
|
||||
- Use `docker logs mctomqtt` for runtime output
|
||||
- Use `docker inspect mctomqtt` for bind mounts, device mappings, and restart policy
|
||||
### MeshCore companion observer
|
||||
- Current runtime: `meshcore-capture.service` (systemd)
|
||||
- Install root: `/home/fizzlepoof/.meshcore-packet-capture`
|
||||
- Service unit: `/etc/systemd/system/meshcore-capture.service`
|
||||
- Runtime command: `/home/fizzlepoof/.meshcore-packet-capture/venv/bin/python3 /home/fizzlepoof/.meshcore-packet-capture/packet_capture.py`
|
||||
- Connection mode: direct serial on the Heltec via `/dev/serial/by-id/usb-Espressif_Systems_heltec_wifi_lora_32_v4__16_MB_FLASH__2_MB_PSRAM__441BF670B684-if00`
|
||||
- Observer role:
|
||||
- NOMAD now uses the LetsMesh companion observer flow instead of the legacy `mctomqtt` relay
|
||||
- The companion process connects to the Heltec locally, signs JWTs on-device, and publishes packet capture data upstream to MQTT brokers
|
||||
- Service health observed on 2026-07-03:
|
||||
- `systemctl status meshcore-capture` shows the service active/running
|
||||
- Logs confirm successful serial connection to device name `Peachtree Mesh Monitor`
|
||||
- Logs confirm MQTT connectivity to LetsMesh US, LetsMesh EU, and NashMe
|
||||
- After removing the temporary PD embedded-broker test target, startup logs now show `Connected to 3 MQTT broker(s)`
|
||||
- Packet capture mode is enabled and waiting for packets
|
||||
- Initial-start caveat:
|
||||
- One installer-start attempt failed with `No such file or directory` for the `/dev/serial/by-id/...` path
|
||||
- A subsequent manual `systemctl start meshcore-capture` succeeded once the device path was present again
|
||||
- Legacy relay retirement:
|
||||
- The old standalone Docker container `mctomqtt` was retired on 2026-06-13
|
||||
- Backup of the retired relay files/config was saved under `/home/fizzlepoof/private/backups/mctomqtt-retire/20260613-174536`
|
||||
- Documentation linkage:
|
||||
- Related mesh stack notes live in [MESHTASTIC.md](../reference/MESHTASTIC.md)
|
||||
|
||||
|
||||
@@ -47,9 +47,10 @@ Recommended models for 11GB VRAM:
|
||||
| `phi4` | ~8GB | Analytical tasks |
|
||||
|
||||
Current light-tier notes:
|
||||
- PD Ollama is the active light-tier endpoint for LiteLLM on `http://10.5.30.6:11434`
|
||||
- Local Honcho currently points at PD Ollama over LAN (`http://10.5.30.6:11434/v1`) instead of using a separate local model host
|
||||
- Current Honcho routing split keeps embeddings on `nomic-embed-text:latest`, cheap/default paths on `qwen2.5:7b`, and deeper high/max paths on `qwen2.5:14b`
|
||||
- PD Ollama remains the shared light-tier endpoint for the broader AI stack on `http://10.5.30.6:11434`, but it should be protected from routine Honcho background load.
|
||||
- Local Honcho now uses N.O.M.A.D.'s own Ollama on `http://127.0.0.1:11434/v1` so PD pressure does not take down MeshMonitor or shared Postgres.
|
||||
- Current Honcho routing keeps embeddings on `nomic-embed-text:v1.5`, cheap/default paths on `qwen2.5:3b`, and high/max paths on `llama3.1:latest`.
|
||||
- Rocinante is John's PC and should be treated as an opportunistic/manual heavy tier, not a dependable always-on backend.
|
||||
|
||||
## Remote Access
|
||||
- Pangolin VPS + Newt for external services
|
||||
|
||||
@@ -24,15 +24,15 @@ Ollama runs on port `11434` and hosts the large-model ("heavy") tier.
|
||||
| `qwen2.5:32b` | Large-context tasks |
|
||||
| `gemma3:27b` | Multimodal / vision |
|
||||
|
||||
> **LiteLLM tier:** Rocinante is the **"heavy"** tier in LiteLLM `config.yaml`.
|
||||
> Heavy-tier requests from OpenWebUI and n8n route to `http://10.5.30.112:11434`.
|
||||
> **LiteLLM tier:** Rocinante may serve as an **opportunistic heavy/manual tier** when the PC is online.
|
||||
> Do not depend on `http://10.5.30.112:11434` for routine background routing; verify live reachability first and prefer N.O.M.A.D.-local paths for guaranteed availability.
|
||||
|
||||
## Current operational note
|
||||
|
||||
- VLAN access to Rocinante from N.O.M.A.D. was re-validated on 2026-05-27.
|
||||
- Rocinante Ollama now answers from N.O.M.A.D. at `http://10.5.30.112:11434`; `/api/tags`, `/api/generate`, and OpenAI-compatible `/v1/chat/completions` all succeeded.
|
||||
- Hermes configs that still pointed at the legacy `10.5.1.112` endpoint were corrected to `10.5.30.112`.
|
||||
- Re-check CUDA Whisper on `http://10.5.30.112:8787/health` separately if Whisper-specific reachability is needed; this pass validated Ollama end-to-end.
|
||||
- Rocinante is John's personal PC and is not a dependable always-on homelab tier.
|
||||
- Treat Ollama/Whisper on Rocinante as opportunistic capacity when the machine is online, not as a required background dependency.
|
||||
- If N.O.M.A.D. or PD need guaranteed inference, prefer local N.O.M.A.D. routing first and keep PD protected from routine background load.
|
||||
- Re-check live reachability from N.O.M.A.D. before any Rocinante-dependent maintenance or cutover.
|
||||
|
||||
## Notes
|
||||
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
import hashlib
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
from urllib.parse import quote_plus
|
||||
|
||||
from fastapi import APIRouter, File, Form, HTTPException, Request, UploadFile
|
||||
from fastapi.responses import RedirectResponse
|
||||
@@ -40,6 +41,11 @@ def _confirm_submission_to_paperless(current: dict) -> dict:
|
||||
if value:
|
||||
fields[optional_key] = value
|
||||
artifact_kind = handoff.get("artifact_kind") or "document"
|
||||
response: dict[str, Any]
|
||||
body_obj: dict[str, Any]
|
||||
ok = False
|
||||
error_detail = None
|
||||
try:
|
||||
if artifact_kind == "source_bundle":
|
||||
paperless_response = paperless.upload_source_bundle_receipt(current)
|
||||
enrichment = paperless.enrich_direct_submission(current, paperless_response)
|
||||
@@ -67,6 +73,17 @@ def _confirm_submission_to_paperless(current: dict) -> dict:
|
||||
body = response.get("body", {}) if isinstance(response, dict) else {}
|
||||
body_obj = body[0] if isinstance(body, list) and body and isinstance(body[0], dict) else (body if isinstance(body, dict) else {})
|
||||
ok = bool(response.get("ok", False)) if isinstance(response, dict) else False
|
||||
if not ok:
|
||||
error_detail = str(response.get("error") or response.get("body") or "Paperless confirmation failed.")
|
||||
except Exception as exc:
|
||||
error_detail = f"{type(exc).__name__}: {exc}"
|
||||
response = {
|
||||
"ok": False,
|
||||
"status_code": 500,
|
||||
"mode": "direct-paperless" if artifact_kind == "source_bundle" else "webhook",
|
||||
"error": error_detail,
|
||||
}
|
||||
body_obj = {}
|
||||
updates = {
|
||||
"status": "paperless-queued" if ok else "paperless-error",
|
||||
"confirmed_at": store.now(),
|
||||
@@ -76,6 +93,7 @@ def _confirm_submission_to_paperless(current: dict) -> dict:
|
||||
"paperless_task_id": body_obj.get("paperless_task_id"),
|
||||
}
|
||||
updated = repo.update_submission(str(current.get("id")), updates) or current
|
||||
updated["paperless_error_detail"] = error_detail
|
||||
grade_note = None
|
||||
if ok and updated.get("assignment_id"):
|
||||
grade_note = paperless.append_grade_note_if_available(str(updated.get("assignment_id")))
|
||||
@@ -87,6 +105,7 @@ def _confirm_submission_to_paperless(current: dict) -> dict:
|
||||
"webhook": [response],
|
||||
"grade_note": grade_note,
|
||||
"file_count": handoff.get("file_count") or 1,
|
||||
"error_detail": error_detail,
|
||||
}
|
||||
|
||||
|
||||
@@ -220,7 +239,10 @@ async def intake_assignment(
|
||||
"backend": repo.backend,
|
||||
}
|
||||
if "text/html" in accept:
|
||||
redirect_url = "/assignments/upload?submitted=" + str(created["id"]) + "&file_count=" + str(bundle["file_count"]) + "&artifact_kind=" + str(created.get("artifact_kind", "document")) + "&paperless_status=" + str(created.get("status", "uploaded-local"))
|
||||
status_value = str(created.get("status", "uploaded-local"))
|
||||
redirect_url = "/assignments/upload?submitted=" + str(created["id"]) + "&file_count=" + str(bundle["file_count"]) + "&artifact_kind=" + str(created.get("artifact_kind", "document")) + "&paperless_status=" + status_value
|
||||
if auto_confirmed and auto_confirmed.get("error_detail"):
|
||||
redirect_url += "&paperless_error=" + quote_plus(str(auto_confirmed["error_detail"]))
|
||||
return RedirectResponse(url=redirect_url, status_code=303)
|
||||
return response_payload
|
||||
|
||||
|
||||
@@ -1,12 +1,19 @@
|
||||
{% extends 'base.html' %}
|
||||
{% block content %}
|
||||
{% if request.query_params.get('submitted') %}
|
||||
<section class="card form-card notice-card success-card evidence-board">
|
||||
<div class="case-legend"><span class="section-label">Submission packet</span><span class="pill">Filed</span></div>
|
||||
<h2>Submission Created</h2>
|
||||
{% set paperless_status = request.query_params.get('paperless_status', 'uploaded-local') %}
|
||||
<section class="card form-card notice-card {{ 'success-card' if paperless_status == 'paperless-queued' else '' }} evidence-board">
|
||||
<div class="case-legend"><span class="section-label">Submission packet</span><span class="pill">{{ 'Filed' if paperless_status == 'paperless-queued' else 'Saved locally' }}</span></div>
|
||||
<h2>{{ 'Submission Created' if paperless_status == 'paperless-queued' else 'Submission Saved, Archive Pending' }}</h2>
|
||||
<p>Submission {{ request.query_params.get('submitted') }} was created successfully.</p>
|
||||
<p class="muted">Artifact: {{ request.query_params.get('artifact_kind', 'document') }} · Files: {{ request.query_params.get('file_count', '1') }}</p>
|
||||
<p class="muted">Paperless status: {{ request.query_params.get('paperless_status', 'uploaded-local') }}</p>
|
||||
<p class="muted">Paperless status: {{ paperless_status }}</p>
|
||||
{% if paperless_status != 'paperless-queued' %}
|
||||
<p class="muted">Schoolhouse kept the submission locally, but the archive handoff failed this time.</p>
|
||||
{% if request.query_params.get('paperless_error') %}
|
||||
<p class="muted">Error: {{ request.query_params.get('paperless_error') }}</p>
|
||||
{% endif %}
|
||||
{% endif %}
|
||||
</section>
|
||||
{% endif %}
|
||||
<section class="card form-card evidence-board">
|
||||
|
||||
@@ -5,10 +5,13 @@ services:
|
||||
ports:
|
||||
- "8081:3001"
|
||||
- "4404:4404"
|
||||
- "1883:1883"
|
||||
restart: unless-stopped
|
||||
volumes:
|
||||
- /mnt/tank/docker/appdata/meshmonitor/data:/data
|
||||
- /mnt/docker-ssd/docker/compose/meshtastic/scripts:/data/scripts
|
||||
devices:
|
||||
- /dev/ttyACM0:/dev/ttyACM0
|
||||
env_file: .env
|
||||
environment:
|
||||
- NODE_ENV=production
|
||||
@@ -68,28 +71,6 @@ services:
|
||||
networks:
|
||||
- default
|
||||
- pangolin
|
||||
mqtt-proxy:
|
||||
image: ghcr.io/ln4cy/mqtt-proxy:master
|
||||
container_name: meshmonitor-mqtt-proxy
|
||||
restart: unless-stopped
|
||||
environment:
|
||||
- INTERFACE_TYPE=tcp
|
||||
- TCP_NODE_HOST=meshmonitor
|
||||
- TCP_NODE_PORT=4404
|
||||
- LOG_LEVEL=INFO
|
||||
- TCP_TIMEOUT=300
|
||||
- CONFIG_WAIT_TIMEOUT=60
|
||||
- HEALTH_CHECK_ACTIVITY_TIMEOUT=300
|
||||
depends_on:
|
||||
- meshmonitor
|
||||
healthcheck:
|
||||
test: ["CMD-SHELL", "test -f /tmp/healthy && find /tmp/healthy -mmin -1 | grep -q healthy"]
|
||||
interval: 30s
|
||||
timeout: 10s
|
||||
retries: 3
|
||||
start_period: 60s
|
||||
networks:
|
||||
- default
|
||||
networks:
|
||||
ix-databases_shared-databases:
|
||||
external: true
|
||||
|
||||
@@ -171,10 +171,16 @@ curl -X PUT http://10.5.30.6:6333/collections/knowledge_base \
|
||||
|
||||
(768 dimensions matches `nomic-embed-text`. If you use a different embedding model, adjust the size.)
|
||||
|
||||
### Pull nomic-embed-text into Ollama (required for workflows 02, 05, 08)
|
||||
### Pull `nomic-embed-text` into the Ollama host backing these workflows (required for workflows 02, 05, 08)
|
||||
|
||||
For N.O.M.A.D.-local workflows / Honcho-style routing:
|
||||
```bash
|
||||
curl http://10.5.30.6:11434/api/pull -d '{"name": "nomic-embed-text"}'
|
||||
curl http://127.0.0.1:11434/api/pull -d '{"name": "nomic-embed-text:v1.5"}'
|
||||
```
|
||||
|
||||
For PD-hosted shared workflows, use the PD endpoint explicitly instead:
|
||||
```bash
|
||||
curl http://10.5.30.6:11434/api/pull -d '{"name": "nomic-embed-text:latest"}'
|
||||
```
|
||||
|
||||
### Paperless Webhook (required for workflows 03, 05, 17, and 19)
|
||||
|
||||
@@ -27,6 +27,12 @@ Basic verification after updates:
|
||||
- optional functional scrape smoke test:
|
||||
- POST `{"url":"https://example.com"}` to `http://127.0.0.1:3302/v1/scrape`
|
||||
|
||||
Firecrawl stability note on PD:
|
||||
- The upstream `harness.js --start-docker` launcher fans out into API + queue worker + extract worker + NUQ workers.
|
||||
- On PD, the default fan-out has caused first-start flaps (`Port 3002 did not become available within 60000ms`) and steady `WORKER STALLED` / `Can't accept connection due to RAM/CPU load` messages.
|
||||
- Keep the PD compose defaults conservative unless the host is re-sized: `NUM_WORKERS_PER_QUEUE=1`, `CRAWL_CONCURRENT_REQUESTS=1`, `MAX_CONCURRENT_JOBS=1`, `BROWSER_POOL_SIZE=1`, `NUQ_WORKER_COUNT=1`, and `HARNESS_STARTUP_TIMEOUT_MS=180000`.
|
||||
- After changing those values, force-recreate `firecrawl-api` and confirm `curl -sS http://127.0.0.1:3302/` answers on the first start without an immediate harness timeout.
|
||||
|
||||
Rollback-first notes:
|
||||
- Back up `docker-compose.yaml` before live edits under `/home/truenas_admin/doris-backups/`
|
||||
- Back up `settings.yml` before SearXNG config edits under `/home/truenas_admin/doris-backups/`
|
||||
|
||||
@@ -126,10 +126,11 @@ services:
|
||||
- POSTGRES_HOST=firecrawl-postgres
|
||||
- POSTGRES_PORT=5432
|
||||
- USE_DB_AUTHENTICATION=${USE_DB_AUTHENTICATION:-false}
|
||||
- NUM_WORKERS_PER_QUEUE=${NUM_WORKERS_PER_QUEUE:-4}
|
||||
- CRAWL_CONCURRENT_REQUESTS=${CRAWL_CONCURRENT_REQUESTS:-4}
|
||||
- MAX_CONCURRENT_JOBS=${MAX_CONCURRENT_JOBS:-2}
|
||||
- BROWSER_POOL_SIZE=${BROWSER_POOL_SIZE:-2}
|
||||
- NUM_WORKERS_PER_QUEUE=${NUM_WORKERS_PER_QUEUE:-1}
|
||||
- CRAWL_CONCURRENT_REQUESTS=${CRAWL_CONCURRENT_REQUESTS:-1}
|
||||
- MAX_CONCURRENT_JOBS=${MAX_CONCURRENT_JOBS:-1}
|
||||
- BROWSER_POOL_SIZE=${BROWSER_POOL_SIZE:-1}
|
||||
- NUQ_WORKER_COUNT=${NUQ_WORKER_COUNT:-1}
|
||||
- BULL_AUTH_KEY=${BULL_AUTH_KEY}
|
||||
- TEST_API_KEY=${TEST_API_KEY}
|
||||
- LOGGING_LEVEL=${LOGGING_LEVEL:-info}
|
||||
@@ -138,7 +139,7 @@ services:
|
||||
- SEARXNG_CATEGORIES=${SEARXNG_CATEGORIES:-}
|
||||
- ALLOW_LOCAL_WEBHOOKS=${ALLOW_LOCAL_WEBHOOKS:-false}
|
||||
- BLOCK_MEDIA=${BLOCK_MEDIA:-true}
|
||||
- HARNESS_STARTUP_TIMEOUT_MS=60000
|
||||
- HARNESS_STARTUP_TIMEOUT_MS=${HARNESS_STARTUP_TIMEOUT_MS:-180000}
|
||||
- NUQ_RABBITMQ_URL=amqp://firecrawl-rabbitmq:5672
|
||||
command: node dist/src/harness.js --start-docker
|
||||
|
||||
|
||||
Reference in New Issue
Block a user