Red teaming
Also: Adversarial Testing
Targeted attack and misuse tests that find weaknesses in an AI system before deployment.
A “red team” deliberately tries to make a system produce errors, harmful answers or data leaks. The findings feed into training, guardrails and configuration.
Major providers test new models this way before release. Your own AI applications – such as a public customer chat – should also undergo such testing before going live.
General information, not legal or professional advice.