Manipulate one attention head step by step.
Select a query token and see how pre-softmax scores become attention weights. Toggle causal masking, change how concentrated the distribution is, and edit a V value to see the resulting output move.
An attention weight alone does not establish why the model produced an output.
Scaled dot-product attention. In a real Transformer, each position produces query, key and value vectors through learned projections. Scores come from QKᵀ and are divided by √dₖ before masking and softmax.
What this tool simulates. To keep every relationship interpretable, the three heads generate a synthetic score matrix S directly at the stage corresponding to QKᵀ/√dₖ. These are not weights from a specific model. Edit the selected row to see exactly what masking and normalization do.
Causal masking. In an autoregressive decoder, scores for future positions become −∞ before softmax. Their final probability is therefore exactly zero.
Values. Real V vectors are high-dimensional. Here each token gets one editable scalar so the final operation stays visible without inventing semantic coordinates: output is the weighted sum Σ αⱼVⱼ.
Multi-head attention. A real layer runs multiple heads with distinct learned projections, then concatenates and projects their outputs. The three synthetic patterns show how heads can produce different distributions; they do not reproduce a trained layer.
Interpretation. Visualizing α shows which positions a head mixes, but that alone does not justify a causal attribution for the final prediction. Jain and Wallace demonstrated that very different attention distributions can yield equivalent predictions; Wiegreffe and Pinter later argued that explanatory usefulness depends on how “explanation” is defined and tested. This tool therefore treats attention as an inspectable internal operation, not as an automatic causal explanation.
Temperature T is an additional educational control; it does not replace the Transformer's original 1/√dₖ scaling. The tool also omits dropout, rotary/relative position biases, GQA/MQA, FlashAttention and other implementation details.
Sources: Vaswani et al. · Attention Is All You Need for scaled dot-product and multi-head attention; PyTorch · scaled_dot_product_attention for current causal-mask semantics; Jain & Wallace · Attention is not Explanation and Wiegreffe & Pinter · Attention is not not Explanation for the interpretability debate. Verified 2026-08-21.