Tool
Will it fit?
How much VRAM a model actually needs — weights, KV cache and optimizer state broken out — and how many consumer or datacenter GPUs that works out to. Tuned for the case the generic calculators handle worst: multiple 24 GB cards.
Model
Params (B)
Layers
Hidden size
Attn heads
KV heads
Precision
Task
Context
Batch
Target GPU
Weights and KV cache are computed exactly from the model config; the activation slice is a coarse estimate that varies with framework and attention kernel. Real usage runs a little higher than the sum — leave headroom. QLoRA forces the frozen base to 4-bit regardless of the precision picker.
The math
Nothing up the sleeve.
- Weights params × bytes-per-weight. FP16 is 2 bytes, INT8/FP8 is 1, INT4/NF4 is 0.5. A 70B model at 4-bit is ~35 GB before anything else.
- KV cache — the long-context killer 2 × layers × kv-heads × head-dim × context × batch × 2 bytes. Grows linearly with context, and at 128k it can dwarf the weights. GQA (few KV heads) is what keeps it survivable.
- Optimizer state (training only) Full fine-tuning with Adam needs ~16 bytes/param — weights, gradients, fp32 master copy and two Adam moments. It's why full FT wants many times the inference memory. LoRA and QLoRA only pay this on the tiny adapters.
- How many GPUs Total ÷ (VRAM × 0.92), rounded up. The 0.92 is because CUDA context, fragmentation and the framework never leave you the whole card.
Next step
Know the size — now pick the method and the engine.
If it fits, the other two explorers cover what to do with it: which fine-tuning method for your constraint, and which serving engine for your hardware.