Red Teaming
How to test the full causal path from adversarial input to authorization, external effect, recovery, and a reproducible release regression.
Links containing ?t= open the video at a specific second.
Video summary
The ideas to retain
The threat model comes before the benchmark
Before choosing a dataset or metric, an evaluation should define at least:
Separate what the model can do from what the system executes
An evaluation should separate at least three levels:
Testing the model and testing the product are different experiments
When measuring base capability, it can make sense to evaluate configurations with mitigations reduced or disabled. The goal is not to deploy them, but to avoid confusing “the system…
Next step
