About this course
As organizations move from experimenting with large language models to deploying LLM-powered applications and autonomous AI agents in production, a new and fast-evolving category of security risk has emerged. This course is built for developers, security engineers, and technical teams responsible for building, integrating, or securing LLM applications and agentic AI workflows, grounded in the OWASP Top 10 for LLM Applications and the NIST AI Risk Management Framework.
The OWASP Top 10 for LLM Applications (2025 edition) is the most widely referenced framework for understanding these risks, and it reflects how quickly the threat landscape has shifted with the rise of agentic AI. Prompt injection remains the top-ranked risk: because LLMs process instructions and data through the same channel, an attacker can craft input the model follows as a command rather than treats as content, and the model often cannot reliably tell the difference. Sensitive information disclosure, where a model memorizes and can be prompted to reveal training data, proprietary information, or other confidential content, rose to the second-ranked risk in the 2025 update.
The most significant recent shift in this space is the rise of agentic AI: systems where an LLM doesn't just generate text but takes autonomous actions, calls tools, or makes decisions with real-world effects. This introduces the risk OWASP calls excessive agency, where a system grants an AI model too much functionality, permission, or autonomy, so a manipulated or malfunctioning model can take unintended and potentially harmful actions rather than simply producing bad output. Other risks covered include supply chain risk from third-party models, datasets, and plugins; data and model poisoning; improper handling of model output before it reaches downstream systems; system prompt leakage; vector and embedding weaknesses in retrieval-augmented systems; and unbounded resource consumption.
This course also applies the NIST AI RMF's four functions, Govern, Map, Measure, and Manage, to explain how organizations should structure AI risk decisions: who approves high-risk AI use cases, how third-party models are vetted before adoption, and how identified risks are tracked and mitigated over an AI system's lifecycle, rather than treated as a one-time review.
This course is designed for developers, architects, and security professionals building or securing LLM-powered applications and AI agents. After completing it, learners will be able to identify the top risks in the OWASP Top 10 for LLM Applications, apply input and output validation to reduce prompt injection and improper output handling risk, design AI agent permissions with least privilege to limit excessive agency, and evaluate third-party AI models and plugins for supply chain risk.
Topics covered
- Prompt injection and sensitive information disclosure
- Excessive agency and securing autonomous AI agents
- Supply chain risk in third-party models, datasets, and plugins
- Data and model poisoning, and improper output handling
- System prompt leakage and vector/embedding weaknesses
- Applying the NIST AI RMF's Govern, Map, Measure, Manage functions
Want this tailored to your organization's policies? Build your own with AI or book a demo.