41 lines
1.5 KiB
Markdown
41 lines
1.5 KiB
Markdown
# Rocinante
|
|
|
|
Heavy AI inference server.
|
|
|
|
> Planned direction: Rocinante becomes optional if the upgraded PD build receives the 4090. If PD does not get the 4090, Rocinante remains the heavy-inference specialist.
|
|
|
|
## Hardware
|
|
|
|
| Component | Details |
|
|
|-----------|---------|
|
|
| **IP** | 10.5.30.112 |
|
|
| **GPU** | NVIDIA RTX 4090 (24 GB VRAM) |
|
|
| **Role** | Heavy AI inference (Ollama) |
|
|
|
|
## Services
|
|
|
|
### Ollama
|
|
|
|
Ollama runs on port `11434` and hosts the large-model ("heavy") tier.
|
|
|
|
| Model | Use |
|
|
|-------|-----|
|
|
| `qwen3:latest` | Fast reasoning / general chat |
|
|
| `qwen2.5:32b` | Large-context tasks |
|
|
| `gemma3:27b` | Multimodal / vision |
|
|
|
|
> **LiteLLM tier:** Rocinante may serve as an **opportunistic heavy/manual tier** when the PC is online.
|
|
> Do not depend on `http://10.5.30.112:11434` for routine background routing; verify live reachability first and prefer N.O.M.A.D.-local paths for guaranteed availability.
|
|
|
|
## Current operational note
|
|
|
|
- Rocinante is John's personal PC and is not a dependable always-on homelab tier.
|
|
- Treat Ollama/Whisper on Rocinante as opportunistic capacity when the machine is online, not as a required background dependency.
|
|
- If N.O.M.A.D. or PD need guaranteed inference, prefer local N.O.M.A.D. routing first and keep PD protected from routine background load.
|
|
- Re-check live reachability from N.O.M.A.D. before any Rocinante-dependent maintenance or cutover.
|
|
|
|
## Notes
|
|
|
|
- No SSH config needed for routine use — Ollama API is accessed over LAN only
|
|
- Do not run multiple large VRAM allocations simultaneously on the 4090
|