--- title: KV Cache & Context Window Explorer description: Visualize how Transformer KV-cache memory grows with context, concurrency, GQA/MQA and precision, then compare it with a memory budget. keywords: LLM KV cache, context window, LLM context memory, GQA, MQA, multi-head attention, inference memory hide: - toc - navigation - footer ---
Tools · Infrastructure · 04

See how much context fits in the KV cache.

KV-cache memory grows linearly with resident tokens and concurrent sequences. Change attention geometry, precision and the memory budget to see when context becomes the constraint.

Contexttokens × sequences
Per tokenlayers × K/V × KV heads
GQA/MQAfewer KV than query heads

Architecture

Presets use architecture geometry published by Meta. Every field remains editable.
GQA/MQA reduces this value relative to the query-head count.

Resident workload

Memory available only for KV cache after weights and runtime reserve have been accounted for.
Selected KV cache—context × concurrency
KV / token / sequence—marginal cost of one resident token
Resident tokens—context × sequences
Budget utilization—of the configured KV-only budget
—
—
—
How KV-cache memory scaleslogarithmic X axis
Selected geometry Full MHA KV budget Preset maximum
Memory-limited max context—
Memory-limited max sequences—
KV versus MHA—
Saving versus MHA—
Equivalent full-MHA KV—
Head dimension—
Method

KV-cache memory scales with the tokens that remain resident.

Per token. In standard decoder attention, every layer retains one key and one value per KV head. Head dimension is hidden_size / attention_heads.

KV_token ≈ layers × 2 × KV_heads × head_dim × KV_bytes

Context and concurrency. Total memory then multiplies by the number of tokens in each sequence and by the number of sequences resident at the same time.

KV_total ≈ KV_token × context_tokens × concurrent_sequences

GQA and MQA. Fewer KV heads reduce cache memory in the same proportion, independently of precision or context length. The chart compares the selected geometry with full MHA using the same query-head count.

Budget. The memory limit applies only to KV cache. It is not total VRAM: weights, activations, CUDA graphs, buffers and other reserves must be subtracted first. Use the AI Inference VRAM Calculator to model the full deployment budget.

This approximation does not model MLA, hybrid/sliding-window attention, recurrent state, backend-specific paging/fragmentation or compressed caches. Validate those deployments against real serving-engine metrics.

Primary sources: Meta · Llama architecture definitions, Meta · Llama 3.1 model card, and vLLM · KV-cache configuration. Verified 2026-08-21.

01Total inference VRAMAdd weights and runtime reserve around the KV-cache budget.Tool 02How the Transformer worksAttention, heads and representations that determine cache geometry.Concept