Start from a model and training-token count, keep or scale the total budget, and inspect which allocation minimizes the loss predicted by the Hoffmann et al. fitted surface. Change the exponents to see how sensitive that conclusion is to the fit.
—
Loss with the same N:D ratio: —
Loss at the optimum: —
Difference attributable to allocation inside this model: —
Tokens per parameter: — now · — at the optimum.
Nopt ∝ C—
Dopt ∝ C—
—
The tool uses L(N,D)=E+A/Nα+B/Dβ. More parameters reduce one term; more data reduce the other.
The approximation C≈6ND turns a fixed budget into a trade-off: if N grows, D must fall to keep the same training cost.
Under that constraint, Nopt scales as Cβ/(α+β) and Dopt as Cα/(α+β). Editable exponents expose how sensitive that conclusion is.
The original fit describes pretraining loss for dense Transformers in a specific experimental regime. It does not include post-training, MoE routing, data quality, architectural changes, inference, RL, tools or serving cost. The coefficients shown here are the rounded values printed in the paper, so an analytical optimum computed from them does not have to reproduce every table or fitted frontier exactly. Use it to understand the parameter–data trade-off and conditional compute optimum, not to quote a real training run.