Stabilize Firecrawl startup on PD
This commit is contained in:
@@ -27,6 +27,12 @@ Basic verification after updates:
|
||||
- optional functional scrape smoke test:
|
||||
- POST `{"url":"https://example.com"}` to `http://127.0.0.1:3302/v1/scrape`
|
||||
|
||||
Firecrawl stability note on PD:
|
||||
- The upstream `harness.js --start-docker` launcher fans out into API + queue worker + extract worker + NUQ workers.
|
||||
- On PD, the default fan-out has caused first-start flaps (`Port 3002 did not become available within 60000ms`) and steady `WORKER STALLED` / `Can't accept connection due to RAM/CPU load` messages.
|
||||
- Keep the PD compose defaults conservative unless the host is re-sized: `NUM_WORKERS_PER_QUEUE=1`, `CRAWL_CONCURRENT_REQUESTS=1`, `MAX_CONCURRENT_JOBS=1`, `BROWSER_POOL_SIZE=1`, `NUQ_WORKER_COUNT=1`, and `HARNESS_STARTUP_TIMEOUT_MS=180000`.
|
||||
- After changing those values, force-recreate `firecrawl-api` and confirm `curl -sS http://127.0.0.1:3302/` answers on the first start without an immediate harness timeout.
|
||||
|
||||
Rollback-first notes:
|
||||
- Back up `docker-compose.yaml` before live edits under `/home/truenas_admin/doris-backups/`
|
||||
- Back up `settings.yml` before SearXNG config edits under `/home/truenas_admin/doris-backups/`
|
||||
|
||||
Reference in New Issue
Block a user