1.5 KiB
1.5 KiB
Rocinante
Heavy AI inference server.
Planned direction: Rocinante becomes optional if the upgraded PD build receives the 4090. If PD does not get the 4090, Rocinante remains the heavy-inference specialist.
Hardware
| Component | Details |
|---|---|
| IP | 10.5.30.112 |
| GPU | NVIDIA RTX 4090 (24 GB VRAM) |
| Role | Heavy AI inference (Ollama) |
Services
Ollama
Ollama runs on port 11434 and hosts the large-model ("heavy") tier.
| Model | Use |
|---|---|
qwen3:latest |
Fast reasoning / general chat |
qwen2.5:32b |
Large-context tasks |
gemma3:27b |
Multimodal / vision |
LiteLLM tier: Rocinante may serve as an opportunistic heavy/manual tier when the PC is online. Do not depend on
http://10.5.30.112:11434for routine background routing; verify live reachability first and prefer N.O.M.A.D.-local paths for guaranteed availability.
Current operational note
- Rocinante is John's personal PC and is not a dependable always-on homelab tier.
- Treat Ollama/Whisper on Rocinante as opportunistic capacity when the machine is online, not as a required background dependency.
- If N.O.M.A.D. or PD need guaranteed inference, prefer local N.O.M.A.D. routing first and keep PD protected from routine background load.
- Re-check live reachability from N.O.M.A.D. before any Rocinante-dependent maintenance or cutover.
Notes
- No SSH config needed for routine use — Ollama API is accessed over LAN only
- Do not run multiple large VRAM allocations simultaneously on the 4090