How Reasoning Fails
Sycophancy, shortcut learning, specification gaming and cascading failures: the failure modes of reasoning models, how to detect them and how to mitigate them.
Links containing ?t= open the video at a specific second.
Video summary
The ideas to retain
1. Failure types
1.1 Shortcuts — shortcut learning
The model learns to solve a problem using superficial correlations instead of the underlying reasoning. The result is correct on training and evaluation data where those correlations hold,…
1.2 Systematic errors
Models have systematic biases that produce non-random errors on particular categories of input. Some of the best documented are:
Next step
