--- title: AI Inference VRAM Calculator description: Estimate model-weight memory, KV cache, runtime reserve, total VRAM and approximate context capacity for Transformer inference with GQA/MQA. keywords: LLM VRAM calculator, AI inference memory, KV cache, GQA, LLM quantization, context VRAM, GPU LLM hide: - toc - navigation - footer ---
Tools · Infrastructure · 03

Calculate what consumes VRAM during inference.

Separate model weights, KV cache and runtime reserve. Change precision, context length, concurrency and GPU count to see which part of the memory budget is constraining the deployment.

Weightsparameters × bits
KV cachelayers × K/V × heads × tokens
Capacityideal GPU sharding

Architecture

Presets use architecture geometry published by Meta. Every field remains editable.
GQA/MQA reduces this value relative to the total query-head count.

Inference scenario

For example: 16 for BF16/FP16, or 8/4 as a quantized approximation.
Sequences simultaneously resident in the KV cache.

Hardware and reserve

Add scales, metadata or format overhead when you know it.
Explicit reserve for temporary activations, kernels and allocator headroom. It is not a universal constant.
Estimated total VRAM—weights + KV cache + reserve
Weights—includes configured format overhead
KV cache—context × concurrent sequences
Memory per GPU——
—
—
—
Where the memory goestotal estimate
WeightsKV cacheRuntime reserve
KV / token / sequence—
Approx. max context—
Approx. max sequences—
Head dimension—
KV versus MHA—
VRAM headroom—
Method

A planning estimate, not a profiler.

Weights. The minimum approximation is parameter count × bits per stored weight. Quantized formats may add scales, codebooks, padding or other metadata, so format overhead is exposed as a separate input rather than hidden.

weight_memory ≈ parameters × weight_bits / 8 × (1 + format_overhead)

KV cache. For standard decoder attention, each layer retains one key and one value vector per KV head, token and sequence. GQA/MQA can use fewer KV heads than query heads. Meta publishes this geometry for Llama 3.1, while vLLM exposes the KV cache as an explicit GPU-memory budget.

KV ≈ layers × 2 × KV_heads × head_dim × KV_bytes × tokens × sequences

Multiple GPUs. The per-GPU figure assumes ideal, even sharding of both weights and KV cache. It is useful as a planning lower bound, but it does not model replicas, pipeline parallelism, indivisible layers, communication buffers, offload or a specific backend layout.

Runtime reserve. The tool applies an editable percentage to weights + KV. Temporary activations, CUDA graphs, kernels and allocator behavior vary by engine and workload, so no fixed percentage is presented as an observed fact.

The KV formula is not a faithful model for MLA, hybrid/sliding-window attention, recurrent state or architecture-specific compressed caches. In those cases, use the result as a comparison estimate and validate the deployment with real serving-engine metrics.

Primary sources: Meta · Llama architecture definitions, Meta · Llama 3.1 model card, and vLLM · KV-cache configuration. Verified 2026-08-21.

01How the Transformer worksAttention, heads and representations before turning them into a memory budget.Concept 02LLM cost and latencyConnect deployment memory with workload cost and response time.Tool