Course Outline
Introduction to AI Red Teaming
- Grasping the AI threat landscape.
- The role of red teams in AI security.
- Ethical and legal implications.
Adversarial Machine Learning
- Attack vectors: evasion, poisoning, extraction, and inference.
- Creating adversarial examples (e.g., FGSM, PGD).
- Differentiating between targeted and untargeted attacks, along with success metrics.
Evaluating Model Robustness
- Assessing robustness under various perturbations.
- Identifying model blind spots and failure modes.
- Stress testing classification, vision, and NLP models.
Red Teaming AI Pipelines
- Mapping the attack surface: data, model, and deployment layers.
- Exploiting insecure model APIs and endpoints.
- Reverse engineering model behavior and outputs.
Simulation and Tooling
- Utilizing the Adversarial Robustness Toolbox (ART).
- Red teaming using tools like TextAttack and IBM ART.
- Sandboxing, monitoring, and observability utilities.
AI Red Team Strategy and Defense Collaboration
- Formulating red team exercises and objectives.
- Effectively communicating findings to blue teams.
- Embedding red teaming into AI risk management frameworks.
Summary and Future Directions
Requirements
- A solid grasp of machine learning and deep learning architectures.
- Practical experience with Python and ML frameworks such as TensorFlow or PyTorch.
- Familiarity with cybersecurity principles or offensive security methodologies.
Target Audience
- Security researchers.
- Members of offensive security teams.
- Professionals in AI assurance and red teaming.
Testimonials (3)
inventory and identifying the different risk exposures within AI
Gary Cook - Cybersecurity and Information Technology Risk Division
Course - Introduction to AI Trust, Risk, and Security Management (AI TRiSM)
I really enjoyed learning about AI attacks and the tools out there to begin practicing and actively using for security testing. I took a lot of knowledge away which I didn't have at the beginning and the course met what I hoped it would be. My favorite part shown from the training was Comet Browser and was amazed at what it could do. Definitely something will be looking into more. Overall it was a great course and enjoyed learning all OWASP GenAI Top 10.
Patrick Collins - Optum
Course - OWASP GenAI Security
The profesional knolage and the way how he presented it before us