Get in Touch
 Duration 21 hours

Course Outline

Module 1 — Failure Modes in AI Applications

Lab: None — Architecture walkthrough & discussion

Establishing a developer-centric mental model of the attack surface.

Topics:

  • LLM, RAG, and agent architectures from a developer’s perspective
  • The request/response lifecycle of AI features
  • Prompt flow: system, developer, user, and tool messages
  • Entry points for untrusted data into the model (and re-entry points)
  • Trust boundaries owned versus inherited by the developer
  • The semantic rather than syntactic nature of AI attacks
  • Mapping the OWASP LLM Top 10 to code implementations

Key Insight: Any point where untrusted text reaches the model, or where model output enters application code, represents a trust boundary under the developer’s control.

Module 2 — Prompt Injection for Builders

Lab: Lab 01 — 01-Prompt-Injection

The “SQL injection equivalent” for AI, where complete prevention is impossible.

Topics:

  • Direct versus indirect prompt injection
  • Hidden instructions embedded in documents, web pages, or tool outputs
  • Jailbreak techniques and role confusion
  • The importance of separating instructions from data
  • Defensive prompt design (using delimiters, structure, and minimal authority)
  • The limits of prevention and the need for containment strategies

Hands-on Activities:

  • Simulating attacks on one’s own chatbot
  • Bypassing naive filter mechanisms
  • Restructuring prompts to minimize the blast radius

Module 3 — Treating Model Output as Untrusted

Lab: Lab 02 — 02-Output-Handling

The most frequently underestimated bug class.

Topics:

  • Viewing model output as untrusted input for the application
  • Insecure output handling (LLM02): downstream XSS, SSRF, command/SQL injection
  • Prohibiting eval/exec/rendering of raw model output
  • Structured outputs and schema validation
  • Output encoding and allowlists
  • Safe rendering in web/UI contexts

Hands-on Activities:

  • Identifying and remediating insecure output handling vulnerabilities
  • Enforcing JSON schemas on model responses

Module 4 — RAG Security

Lab: Lab 03 — 03-RAG-Security

A significant new attack surface that developers must construct securely.

Topics:

  • Threats to vector databases and retrieval processes
  • Ingestion sanitization techniques
  • Document provenance and trust scoring
  • Retrieval scoping and metadata isolation
  • Hidden instructions in retrieved content (indirect injection)
  • Data exfiltration via retrieval mechanisms

Hands-on Activities: - Inject malicious documents into a RAG pipeline - Implement ingestion sanitization and retrieval scoping to defend against it

Module 5 — Agent & Tool Safety

Lab: Lab 04 — 04-Agent-Safety

The point where logical bugs translate into executed actions.

Topics:

  • Excessive agency (LLM06) and tool abuse
  • Least-privilege principles for agents
  • Tool allowlists and argument validation
  • Approval gates and human-in-the-loop mechanisms
  • Sandboxing tool execution
  • Scoped, short-lived credentials for agents
  • Limiting autonomous loops and chaining

Hands-on Activities:

  • Restricting over-permissioned agents
  • Implementing allowlists and approval gates for dangerous tools

Module 6 — Secrets, Identity & Cost

Lab: Lab 05 — 05-Secrets-and-Cost

Operational errors that cause the most rapid harm.

Topics:

  • API key and secret management (excluded from prompts, code, and logs)
  • Per-user authentication and authorization for AI features
  • Propagating user identity to tools and retrieval systems
  • Denial-of-wallet: unbounded token/cost consumption
  • Rate limits, token budgets, and timeouts
  • Logging without exposing secrets or PII

Hands-on Activities:

  • Removing secrets from the prompt/code path
  • Implementing per-user rate limits and token/cost budgets

Module 7 — Guardrail Libraries

Lab: Lab 06 — 06-Guardrails

Deciding between buying vs. building for input/output safety.

Topics:

  • Capabilities and limitations of guardrail frameworks
  • Input guardrails: injection/PII/topic classifiers
  • Output guardrails: validation, filtering, and grounding checks
  • Determining when guardrails are appropriate versus using custom deterministic checks
  • Integrating guardrails with controls from previous modules
  • Performance, false positives, and failure modes

Hands-on Activities:

  • Integrating input/output guardrail layers into AI features
  • Measuring detection accuracy and identifying gaps

Module 8 — Red-Teaming Your Own App

Lab: Lab 07 — 07-Red-Teaming

Deploying with the assumption that attackers already have access.

Topics:

  • Building abuse/test suites for AI features
  • Automated prompt injection and jailbreak testing
  • Regression testing of guardrails and policies
  • Incorporating AI security checks into CI pipelines
  • Model and dependency supply chain (provenance, pinning)
  • Pre-deployment security checklists for AI features

Hands-on Activities:

  • Writing automated red-team tests for AI features
  • Integrating these tests into CI checks

Module 9 — Scoring AI Security: The SAIS-100 Framework

Lab: None — Scoring exercise (utilizing the Capstone app)

Converting built features into a repeatable security score.

Topics:

  • The AI Security Hexagon: six questions beyond “is it secure?”
  • The six scored categories (Data, Prompt, Agent, Supply Chain, Detection, Governance)
  • The 100-point rubric and its weightings
  • Verdict bands and single-category override rules
  • The Elephant Scale Secure AI Score (SAIS-100) as a branded, repeatable framework
  • Scoring before and after hardening as a performance metric

Hands-on Activities:

  • Scoring the Capstone app on the 100-point scale
  • Identifying the single change that most improves the score

Key Insight: The three highest-weighted categories correspond to the trust boundaries owned by the developer, ensuring the score accurately reflects the course’s core teachings.

Capstone

Students harden a deliberately vulnerable AI application end-to-end.

The starter application includes:

  • An injectable prompt
  • Insecure output handling
  • An unscoped RAG pipeline
  • An over-permissioned agent
  • Secrets within the prompt path
  • Lack of cost limits

Students apply course concepts to:

  • Restructure prompts for containment
  • Validate and encode model output
  • Sanitize and scope retrieval processes
  • Apply least privilege and approval gates to the agent
  • Remove secrets and add cost/rate limits
  • Implement guardrails and automated red-team tests

Deliverable: A hardened application and a concise OWASP LLM Top 10 self-assessment.

Module - Lab Map

Labs are executed in sequence, following the module order. The course comprises 9 modules and 7 labs: Module 1 is an architecture walkthrough/discussion, and Module 9 is a scoring exercise; neither has a dedicated lab folder.

  • Lab 01 - 01-Prompt-Injection: Attacking the chatbot & designing for containment (Module 2)
  • Lab 02 - 02-Output-Handling: Fixing insecure output handling bugs (Module 3)
  • Lab 03 - 03-RAG-Security: Poisoning then defending a RAG pipeline (Module 4)
  • Lab 04 - 04-Agent-Safety: Restricting over-permissioned agents (Module 5)
  • Lab 05 - 05-Secrets-and-Cost: Securing keys + adding cost guardrails (Module 6)
  • Lab 06 - 06-Guardrails: Adding input/output guardrail layers (Module 7)
  • Lab 07 - 07-Red-Teaming: Automated red-team tests in CI (Module 8)

Module 1 (Failure Modes in AI Applications) has no lab — it is conducted as an architecture walkthrough and discussion. Module 9 (Scoring AI Security) has no lab folder — it is conducted as a scoring exercise against the Capstone app.

Requirements

  • Intermediate skill level.
  • Proficiency in building and consuming REST APIs, a scripting language (labs utilize Python), basic application authentication, Git, and command-line interfaces is expected.
  • No machine learning background is required; this is an application security course for developers building with LLMs, not training them.

Target Audience

  • Software / Backend engineers developing LLM features
  • Full-stack and API developers
  • AI/ML application engineers
  • Platform engineers deploying copilots and agents
  • Tech leads and senior engineers responsible for AI features

Number of participants


Price per participant

Testimonials (3)

Upcoming Courses

Related Categories