truenas-stacks/docs/servers/ROCINANTE.md

Rocinante

Heavy AI inference server.

Planned direction: Rocinante becomes optional if the upgraded PD build receives the 4090. If PD does not get the 4090, Rocinante remains the heavy-inference specialist.

Hardware

Component	Details
IP	10.5.30.112
GPU	NVIDIA RTX 4090 (24 GB VRAM)
Role	Heavy AI inference (Ollama)

Services

Ollama

Ollama runs on port 11434 and hosts the large-model ("heavy") tier.

Model	Use
qwen3:latest	Fast reasoning / general chat
qwen2.5:32b	Large-context tasks
gemma3:27b	Multimodal / vision

LiteLLM tier: Rocinante is the "heavy" tier in LiteLLM config.yaml. Heavy-tier requests from OpenWebUI and n8n route to http://10.5.30.112:11434.

Current operational note

• VLAN access to Rocinante from N.O.M.A.D. was re-validated on 2026-05-27.
• Rocinante Ollama now answers from N.O.M.A.D. at http://10.5.30.112:11434; /api/tags, /api/generate, and OpenAI-compatible /v1/chat/completions all succeeded.
• Hermes configs that still pointed at the legacy 10.5.1.112 endpoint were corrected to 10.5.30.112.
• Re-check CUDA Whisper on http://10.5.30.112:8787/health separately if Whisper-specific reachability is needed; this pass validated Ollama end-to-end.

Notes

• No SSH config needed for routine use — Ollama API is accessed over LAN only
• Do not run multiple large VRAM allocations simultaneously on the 4090