truenas-stacks/docs/servers/ROCINANTE.md

Rocinante

Heavy AI inference server.

Hardware

Component	Details
IP	10.5.1.112
GPU	NVIDIA RTX 4090 (24 GB VRAM)
Role	Heavy AI inference (Ollama)

Services

Ollama

Ollama runs on port 11434 and hosts the large-model ("heavy") tier.

Model	Use
qwen3:latest	Fast reasoning / general chat
qwen2.5:32b	Large-context tasks
gemma3:27b	Multimodal / vision

LiteLLM tier: Rocinante is the "heavy" tier in LiteLLM config.yaml. Heavy-tier requests from OpenWebUI and n8n route to http://10.5.1.112:11434.

Current operational note

• During Doris validation from N.O.M.A.D. on 2026-05-21, 10.5.1.112 did not answer ping and did not accept connections on 11434 or 8787.
• Treat Rocinante-hosted Ollama heavy-tier traffic and CUDA Whisper as currently unreachable from N.O.M.A.D. until host/network reachability is restored.
• Re-validate from N.O.M.A.D. with curl http://10.5.1.112:11434/api/tags and curl http://10.5.1.112:8787/health after recovery.

Notes

• No SSH config needed for routine use — Ollama API is accessed over LAN only
• Do not run multiple large VRAM allocations simultaneously on the 4090