Fix search stack update recovery notes

search/README.md

@@ -0,0 +1,38 @@
+# Search stack operator notes
+
+Live stack path on PD:
+- `/mnt/docker-ssd/docker/compose/search`
+
+Live SearXNG config path on PD:
+- `/mnt/tank/docker/appdata/searxng/settings.yml`
+
+Known update-sensitive points:
+
+1. Firecrawl containers share Gluetun's network namespace.
+   - `firecrawl-playwright-service` and `firecrawl-api` run with `network_mode: service:search-gluetun`.
+   - If `search-gluetun` is recreated during an update, those containers must be recreated too or they can die with namespace errors like:
+     - `joining network namespace of container ... No such container`
+     - `getaddrinfo EAI_AGAIN firecrawl-redis`
+   - The compose file now uses long-form `depends_on` with `restart: true` so explicit Compose updates to `search-gluetun` also restart the Firecrawl dependents.
+
+2. SearXNG upstream can ship config drift.
+   - After an image update, compare `/mnt/tank/docker/appdata/searxng/settings.yml` against `/mnt/tank/docker/appdata/searxng/settings.yml.new`.
+   - Merge upstream changes deliberately instead of overwriting local behavior blindly.
+   - One confirmed breakage was an obsolete `ask` engine stanza lingering in the live settings after upstream removed that engine module; the fix was to remove the stale stanza and restart `searxng`.
+
+Basic verification after updates:
+- `sudo -n docker compose -f /mnt/docker-ssd/docker/compose/search/docker-compose.yaml ps`
+- `curl -sS "http://127.0.0.1:8888/search?q=test&format=json" | head`
+- `curl -sS http://127.0.0.1:3302/`
+- optional functional scrape smoke test:
+  - POST `{"url":"https://example.com"}` to `http://127.0.0.1:3302/v1/scrape`
+
+Rollback-first notes:
+- Back up `docker-compose.yaml` before live edits under `/home/truenas_admin/doris-backups/`
+- Back up `settings.yml` before SearXNG config edits under `/home/truenas_admin/doris-backups/`
+- If a Compose-side update breaks the stack and quick repair fails, restore the previous compose/config files and `docker compose up -d` the search stack again.
+
+AI maintenance flow:
+- Homelab Docker maintenance is seeded into the Hermes `homelab` kanban board on a daily cadence.
+- The maintenance lane should treat this stack as update-sensitive and must run the verification commands above after any search-stack image change.
+- If the stack cannot be repaired quickly after an update, roll back first, then leave the exact failure evidence in the kanban task so alerts route cleanly.

search/docker-compose.yaml

@@ -78,8 +78,11 @@ services:
     image: ghcr.io/firecrawl/playwright-service:latest
     container_name: firecrawl-playwright-service
     restart: unless-stopped
+    # Shares gluetun's network namespace; restart with gluetun updates so the namespace stays valid.
     depends_on:
-      - search-gluetun
+      search-gluetun:
+        condition: service_started
+        restart: true
     network_mode: "service:search-gluetun"
     environment:
       - PORT=3000
@@ -93,17 +96,20 @@ services:
     image: ghcr.io/firecrawl/firecrawl:latest
     container_name: firecrawl-api
     restart: unless-stopped
+    # Shares gluetun's network namespace; compose should restart it when gluetun/playwright are recreated.
     depends_on:
       firecrawl-redis:
         condition: service_started
       firecrawl-playwright-service:
         condition: service_started
+        restart: true
       firecrawl-rabbitmq:
         condition: service_healthy
       firecrawl-postgres:
         condition: service_started
       search-gluetun:
         condition: service_started
+        restart: true
     network_mode: "service:search-gluetun"
     environment:
       - HOST=0.0.0.0