truenas-stacks/docs/servers/ROCINANTE.md

Rocinante

Heavy AI inference server.

Planned direction: Rocinante becomes optional if the upgraded PD build receives the 4090. If PD does not get the 4090, Rocinante remains the heavy-inference specialist.

Hardware

Component	Details
IP	10.5.30.112
GPU	NVIDIA RTX 4090 (24 GB VRAM)
Role	Heavy AI inference (Ollama)

Services

Ollama

Ollama runs on port 11434 and hosts the large-model ("heavy") tier.

Model	Use
qwen3:latest	Fast reasoning / general chat
qwen2.5:32b	Large-context tasks
gemma3:27b	Multimodal / vision

LiteLLM tier: Rocinante is the "heavy" tier in LiteLLM config.yaml. Heavy-tier requests from OpenWebUI and n8n route to http://10.5.30.112:11434.

Current operational note

• During Doris validation from N.O.M.A.D. on 2026-05-21, 10.5.30.112 did not answer ping and did not accept connections on 11434 or 8787.
• Treat Rocinante-hosted Ollama heavy-tier traffic and CUDA Whisper as currently unreachable from N.O.M.A.D. until host/network reachability is restored.
• Re-validate from N.O.M.A.D. with curl http://10.5.30.112:11434/api/tags and curl http://10.5.30.112:8787/health after recovery.

Notes

• No SSH config needed for routine use — Ollama API is accessed over LAN only
• Do not run multiple large VRAM allocations simultaneously on the 4090