Jailbreaks, adversarial inputs and guardrails
A · 10-second intuition A jailbreak is an input that gets a model to do what its safety training says it should refuse; guardrails are extra layers that try to catch that.
Predict before you read
Why is a single failed jailbreak attempt weak evidence of robustness?