Remuda
Understanding memory
What the runtime chips mean, and where the fit predictor's numbers come from.
The runtime chips
While a model is loaded, the top bar and the load pane show what ollama ps knows:
a all on GPU · 5.6 GB chip when the whole thing fits in VRAM, or an amber
28.4 GB GPU + 9.1 GB RAM chip the moment any of it spills into
system RAM. A model fully on GPU and the same model spilling behave like different products — a
spill costs a large drop in tokens per second — so the split is shown up front rather than folded
into one green dot.
Each chip opens a popover with more detail: the context the runner was started with against the
model's trained maximum, and a live countdown to when keep_alive expires
(never, for an infinite one). This readout always describes whatever is actually loaded in
Ollama — browsing a different model in the load pane never changes what the chips report.
The fit predictor
Before you load a model, the load pane can estimate whether it — and a chosen context length —
will fit in the GPU's share of memory, so a load that's going to spill is visible before you wait
for it. It's built from figures Ollama already reports: the model's architecture
(/api/show), its installed weight size, and how much memory the host actually has.
The fit figure is an estimate, not a measurement. Two things it can't know in advance:
- It assumes an f16 KV cache — the size Ollama uses by default. If you run a
quantised cache (
OLLAMA_KV_CACHE_TYPE=q8_0or smaller), no endpoint reports that, so the predictor has no way to see it and the estimate runs high. - GPU utilisation is not available on Apple Silicon. There is no supported API for it, so Remuda never shows a GPU-percent figure and the predictor leans on the VRAM/RAM split instead.
Calibration
The first time a model actually loads cleanly on GPU, Remuda compares the real result against what it predicted and remembers the ratio, per model tag. Every later prediction for that tag folds that ratio back in, so the estimate gets closer to reality the more you actually load the model. A spilled or partial load is never used as a calibration point — it isn't a clean reading.
When there's nothing to go on — the server didn't report enough about the architecture, or there is no memory reading at all — the predictor says so plainly rather than showing a number it can't back up.