AISecurity.Academy
Course track · 24 modules

AI Security Fundamentals

Securing models, applications, and AI systems across the lifecycle.

Module 1 in pilot productionFoundation to advanced24 modules · 4 parts

Module 1 scripts, labs, and knowledge check are complete and awaiting recording. Modules 2 to 24 follow the same production standard once the pilot is approved.

Who it is for. Students in cybersecurity and AI programs, AI and ML engineers, and cybersecurity practitioners who need a rigorous account of how AI systems work before they can threat-model, defend, monitor, and assure them. No prior exposure to adversarial machine learning is assumed.

Part I: Foundations of SecureAI

Module 2: SecureAI vs. AI for Cybersecurity

What the module covers. "AI security" is used to name three different things: protecting AI systems from attack, using AI to defend networks and applications, and the offensive use of AI by adversaries. This chapter separates them, explains the mechanisms behind each, and then shows why they intersect: an AI-enabled defense is itself an AI system with every weakness described in Chapter 1, and offensive AI raises the volume and quality of attacks that both AI-based and conventional defenses must withstand. The chapter also introduces the problem of automation and human oversight, which recurs whenever a model's judgment is placed in a decision loop.

Learning objectives (6)
  1. Distinguish security for AI, AI for cybersecurity, and offensive misuse of AI, and give a concrete example of each.
  2. Explain how the principal AI-for-security techniques (classification, anomaly detection, LLM-assisted triage, automated response) work.
  3. Analyze an AI-based defensive tool as an AI system and identify its model-, application-, and system-layer attack surface.
  4. Explain why base rates and nonstationary data make evaluation of AI-based detectors difficult.
  5. Assess the risks that offensive AI creates for an organization and identify controls that do not depend on detecting AI-generated content.
  6. Design a human-oversight arrangement that mitigates automation bias without negating the benefit of automation.

Module 3: Cybersecurity Foundations for AI Systems

What the module covers. AI security is continuous with cybersecurity. The principles that protect ordinary systems apply unchanged to AI systems: least privilege, complete mediation, fail-safe defaults, defense in depth, zero trust, secure by design, threat modeling, and a secure development lifecycle. What changes is where they must be applied and how easily they are forgotten when a model appears to "decide" things. This chapter reviews those foundations for readers coming from AI engineering, and restates them in AI terms for readers coming from security. Every later chapter assumes the vocabulary introduced here.

Learning objectives (7)
  1. Define the core security properties and the classical design principles, and give an AI- system example for each.
  2. Explain the zero-trust model of NIST SP 800-207 and map its logical components onto an AI application.
  3. Distinguish authentication, authorization, and delegation, and explain the confused- deputy problem.
  4. Apply defense in depth and secure-by-design principles to AI system components.
  5. Describe the structure of the Secure Software Development Framework (SSDF) and its generative-AI community profile.
  6. Analyze an AI application for classical web and API weaknesses using OWASP guidance.
  7. Design a least-privilege identity and authorization scheme for a simple AI application.

Module 4: AI and Machine Learning Foundations

What the module covers. This chapter explains the machine learning mechanisms that the rest of the book depends on: what a model is, how it is trained, why it generalizes and why it fails to, how neural networks draw decision boundaries, how transformers turn a context of tokens into a prediction, and how foundation models are adapted to tasks. Each mechanism is followed by a short Security Implication identifying the asset, assumption, or attack surface it creates. The chapter is deliberately a foundations chapter rather than an attack chapter; Part II will refer back to each mechanism here when it explains the attack that exploits it. Readers with a strong ML background should still read the Security Implication subsections, which state the book's position on what each mechanism means for security.

Learning objectives (7)
  1. Distinguish artificial intelligence, machine learning, deep learning, and generative AI, and the supervised, unsupervised, self-supervised, and reinforcement learning paradigms.
  2. Explain the roles of datasets, features, labels, training, validation, testing, and inference, and identify the assets each creates.
  3. Explain loss functions and gradient descent at an intuitive level, and why a model's behavior is determined by its data and objective.
  4. Explain generalization, overfitting, and memorization, and relate them to privacy and robustness.
  5. Describe how neural networks form decision boundaries and why that produces adversarial vulnerability.
  6. Explain tokens, embeddings, attention, the context window, and autoregressive generation in transformer language models.
  7. Describe transfer learning, fine-tuning, and alignment of foundation models, and identify the supply-chain and behavioral assumptions they introduce.

Module 5: The AI System Lifecycle

What the module covers. Security controls that are not attached to a point in a process are not applied. This chapter presents the fifteen-stage AI system lifecycle used throughout the book, explains how the data lifecycle, the model development lifecycle, and the MLOps operations lifecycle interleave within it, and asks the same seven questions at every stage: what assets exist, who can influence them, what trust boundaries exist, what properties can fail, what attack paths are possible, what controls reduce likelihood or impact, and what evidence proves the controls are working. It closes with evaluation gates, change management, monitoring, retirement, and the provenance and documentation practices (model cards, datasheets, lineage) that make the lifecycle auditable. Later chapters attach specific attacks and controls to these stages.

Learning objectives (6)
  1. Describe the fifteen lifecycle stages and explain how the data, model development, and operations lifecycles interleave.
  2. Identify the assets, influencers, and trust boundaries present at each stage.
  3. Analyze a lifecycle stage for the security properties that can fail and the attack paths that can reach it.
  4. Explain the purpose of evaluation gates, release promotion, and change management for AI systems.
  5. Design a provenance and documentation scheme using model cards, datasheets, and lineage records.
  6. Specify the evidence each stage should produce and how long it must be retained.

Part II: The AI Threat Landscape

Module 6: Threat Modeling AI Systems

What the module covers. Threat modeling is the disciplined answer to "what can go wrong?" asked before it does. This chapter adapts established threat-modeling practice (decomposition, asset identification, trust boundaries, adversary modeling, STRIDE, attack trees) to AI systems, where data behaves like instructions, behavior is learned, and outputs become actions. It introduces the adversary model of NIST's adversarial machine learning taxonomy, shows how MITRE ATLAS supplies a shared vocabulary of tactics and techniques, distinguishes attack from misuse and failure so that each is modeled with the right method, and extends the process to retrieval-augmented and agentic systems. The chapter's output is a repeatable method that Chapters 7-12 populate with specific attacks and that Chapter 24's capstone requires.

Learning objectives (7)
  1. Decompose an AI system into components, data flows, trust boundaries, and entry points using a data-flow diagram.
  2. Identify the assets of an AI system, including data, model, context, action, identity, and evidence assets.
  3. Characterize adversaries by goal, capability, and knowledge using the NIST AI 100- 2e2025 taxonomy.
  4. Apply STRIDE and attack trees to AI systems without forcing inappropriate one-to-one mappings.
  5. Map threats to MITRE ATLAS tactics and techniques and use ATLAS to check coverage.
  6. Distinguish attacks, misuse, and failures and select the modeling method appropriate to each.
  7. Produce a threat model for a RAG-and-tools assistant that traces each threat to a control, a test, and evidence.

Module 7: Adversarial Machine Learning and Evasion Attacks

What the module covers. An adversarial example is an input deliberately constructed so that a model misclassifies it while a person would not. The phenomenon was reported in 2013, has been reproduced across the major domains in which deep learning is used, and has resisted every proposed cure. This chapter explains why adversarial examples exist, building directly on the decision-boundary and gradient mechanisms of Chapter 4; distinguishes white-, gray-, and black-box attacks and the transferability that connects them; describes evasion at inference time in classifiers, detectors, and multimodal and language models; and then turns to what defenders can actually do: evaluate robustness honestly, apply adversarial training and certified defenses within their limits, and contain the residual risk at the application and system layers. The chapter's stance is that evasion is a model-layer weakness that cannot be removed and must therefore be managed.

Learning objectives (7)
  1. Explain why adversarial examples exist in terms of decision boundaries, gradients, and high-dimensional input spaces.
  2. Distinguish white-box, gray-box, and black-box evasion and explain transferability.
  3. Describe the principal attack families (FGSM, PGD, C&W, query-based, transfer, physical, and text/multimodal variants) at a conceptual level.
  4. Analyze which perturbation constraints and attacker capabilities are realistic for a given deployment.
  5. Evaluate a robustness claim, identifying the common errors that produce false confidence.
  6. Assess adversarial training, input transformations, detection, and certified defenses, including their costs and limitations.
  7. Design application- and system-layer controls that contain evasion when the model cannot be made robust.

Module 8: Data Poisoning and Training-Time Attacks

What the module covers. If data programs the model, then whoever controls the data can program the model. This chapter examines attacks that exploit that fact: poisoning that degrades a model for everyone, poisoning that misclassifies chosen targets, backdoors that install a hidden trigger, clean-label variants that evade human review, poisoning of fine-tuning and preference data that erodes the safety behavior of language models, and poisoning of the retrieval corpora that generative systems consult at inference time. It explains why these attacks are feasible at web scale, why they survive evaluation and often survive fine-tuning, and why the primary defenses are provenance, validation, and pipeline integrity rather than anything applied to the trained model. The chapter builds on the learning paradigms and decision-boundary mechanisms of Chapter 4 and the lifecycle stages of Chapter 5.

Learning objectives (7)
  1. Distinguish availability poisoning, targeted integrity poisoning, and backdoor attacks, and explain the mechanism of each.
  2. Explain clean-label poisoning and why it defeats label auditing.
  3. Analyze the poisoning surfaces created by web-scale pretraining, fine-tuning, human feedback, and retrieval corpora.
  4. Describe how backdoors persist through fine-tuning and safety training, and the evidence for the number of poison samples required.
  5. Distinguish RAG corpus poisoning from training-time poisoning and explain why it is an operational rather than a model-development problem.
  6. Evaluate detection and mitigation techniques (data validation, spectral and activation analysis, trigger reconstruction, fine-pruning) and their limits.
  7. Design provenance, validation, and pipeline controls that reduce poisoning risk at each lifecycle stage.

Module 9: Model Inversion, Membership Inference, and Privacy Leakage

What the module covers. A trained model is a compressed summary of its training data, and compression is not the same as anonymization. This chapter explains how models leak information about the individuals whose data trained them: membership inference reveals whether a record was used; attribute inference recovers hidden fields; model inversion reconstructs representative inputs; training-data extraction makes generative models emit memorized text or images; and property inference reveals facts about the dataset as a whole. It builds on the generalization and memorization mechanisms of Chapter 4, distinguishes privacy from security while showing where they coincide, introduces differential privacy as the principled defense and explains its costs, and covers the practical controls (data minimization, deduplication, output restriction, monitoring, and unlearning) that make up a privacy-preserving deployment. The application-layer disclosure risks of generative systems (leaking system prompts or retrieved documents) are noted here and treated in Chapter 11.

Learning objectives (7)
  1. Explain why trained models retain information about individual training examples and how overfitting and memorization govern leakage.
  2. Describe membership inference, attribute inference, model inversion, training-data extraction, and property inference, with the access each requires.
  3. Analyze the privacy exposure of a deployment from the outputs it exposes (labels, scores, embeddings, gradients, weights).
  4. Explain differential privacy, including its definition, the role of , and DP-SGD, and assess its accuracy and cost tradeoffs.
  5. Evaluate practical controls at the data, training, serving, and governance layers and their limitations.
  6. Distinguish privacy failures from security failures and identify where a privacy attack is also a security incident.
  7. Design a privacy evaluation, including membership-inference testing and memorization canaries, with evidence.

Module 10: Model Extraction and Intellectual Property Risk

What the module covers. A model exposed through a prediction interface reveals, with every answer, something about the function that produced it. Ask enough well-chosen questions and the answers suffice to train a copy. This chapter explains model extraction: why prediction APIs are an extraction surface, how functional replication works, what distinguishes a high-accuracy copy from a high-fidelity one, and how attackers infer architecture and parameters. It also covers the blunter route (theft of the model artifact itself) and the intellectual-property stakes of both. Because a stolen or replicated model also enables the offline evasion attacks of Chapter 7 and the privacy attacks of Chapter 9, extraction is a gateway attack as well as a theft. Defenses, including output restriction, rate limits, detection, watermarking, and artifact protection, are examined together with their limitations, which are substantial.

Learning objectives (7)
  1. Explain why a prediction API leaks the model and how the outputs it returns determine the cost of extraction.
  2. Distinguish functional replication (accuracy) from fidelity extraction and from parameter or architecture recovery.
  3. Describe the principal extraction methods (equation solving, learning-based distillation, active and adaptive querying) and their query costs.
  4. Analyze how an extracted copy enables downstream evasion and privacy attacks.
  5. Assess artifact-theft risks to model weights across the lifecycle and the side channels that leak architecture.
  6. Evaluate defenses (output restriction, rate limiting, query monitoring, prediction perturbation, watermarking) and their limitations.
  7. Design an extraction-resistant serving configuration with monitoring and evidence.

Module 11: Generative AI, LLMs, Prompt Injection, and RAG Security

What the module covers. Large language models are widely deployed in enterprises, and retrieval-augmented generation (RAG) is a common way of connecting them to organizational data. This chapter first completes the mechanism foundations from Chapter 4 (autoregressive generation, tokens and context windows, the roles of system, user, and tool messages, embeddings, vector search, and context assembly) and then examines the application-layer attacks those mechanisms make possible: direct and indirect prompt injection; jailbreaks, which are distinguished from injection; insecure output handling; sensitive-information disclosure including hidden-context exposure; RAG poisoning and retrieval manipulation; embedding and vector-store risks; and the authorization, tenant-isolation, and citation-integrity failures peculiar to retrieval systems. It also separates hallucination, a reliability failure, from security compromise. The chapter's central claim is that prompt injection is structural and cannot be solved at the model; the application must be designed so that injected instructions cannot do harm.

Learning objectives (7)
  1. Explain autoregressive generation, the context window, message roles, embeddings, vector search, and RAG context assembly.
  2. Distinguish direct prompt injection, indirect prompt injection, and jailbreaks by who the attacker is and what is attacked.
  3. Analyze why prompt injection cannot be reliably prevented at the model layer and where it must be contained.
  4. Identify the output sinks through which injected instructions cause harm and the controls at each.
  5. Describe RAG poisoning, retrieval manipulation, embedding inversion, and cross- tenant retrieval, and the controls for each.
  6. Distinguish hallucination from security compromise and explain why grounding and citation integrity matter to both.
  7. Design a secure RAG architecture with authorization before retrieval, origin labeling, tenant isolation, and output handling, and specify its tests.

Module 12: AI Agents, Tool Use, MCP, and Excessive Agency Risk

What the module covers. An agent is a language model placed in a loop: it reads context, decides on an action, invokes a tool, observes the result, and repeats until a goal is met. That single architectural change, letting the model's output drive actions in the world, converts every attack in Chapter 11 from a content problem into a consequences problem. This chapter explains the agent loop, tool calling, memory, planning, orchestration, and the Model Context Protocol (MCP) at a mechanism level, then examines the security risks that autonomy and delegated authority create: excessive agency, tool misuse, the confused-deputy problem, indirect injection into tools and context, privilege escalation through delegation, memory poisoning, unsafe planning, cross-agent trust failures, and the trust boundaries MCP introduces. Defenses center on a tool broker that mediates every action under the user's identity, least-privilege tools, human approval for consequential actions, reversibility, and audit evidence. The chapter maps throughout to the OWASP Top 10 for Agentic Applications 2026.

Learning objectives (7)
  1. Explain the agent loop, tool calling, memory, planning, and orchestration, and identify the trust boundaries each introduces.
  2. Describe MCP at a protocol-concept level and identify the trust boundaries between host, client, and server.
  3. Define excessive agency and decompose it into excessive functionality, permissions, and autonomy.
  4. Analyze the confused-deputy problem in agents and explain why identity propagation is the core control.
  5. Describe tool-description poisoning, memory poisoning, cross-agent trust failures, and cascading failures.
  6. Design a tool broker with least privilege, approval, reversibility, sandboxing, egress control, and audit evidence.
  7. Evaluate an agentic system against the OWASP Agentic Top 10 and test it with an agent-security benchmark.

Part III: SecureAI Engineering and Defense

Module 13: Secure AI Architecture and Deployment

What the module covers. Parts I and II established that model-layer weaknesses (adversarial examples, prompt injection, memorization) cannot be eliminated, and that the application and system layers must therefore be designed to contain them. This chapter turns those conclusions into architecture. It presents a secure AI reference architecture organized into trust zones with gateways and policy enforcement points; it consolidates the input and output controls, the secure RAG and agent reference designs of Chapters 11 and 12, network segmentation and data boundaries, secrets and key management, and secure fail states (a control often omitted), so that when a component fails or is attacked, the system fails closed and degrades safely rather than catastrophically. The chapter is the architectural synthesis of Part II's defenses and the foundation for the pipeline, identity, and monitoring chapters that follow.

Learning objectives (7)
  1. Design a zoned AI architecture with defense in depth and explicit trust boundaries.
  2. Place policy enforcement points (AI gateway, retrieval PEP, tool broker) correctly and explain what each enforces.
  3. Specify input and output controls and explain why they are layers, not guarantees.
  4. Apply segmentation, data boundaries, and least privilege to model serving, retrieval, and tool execution.
  5. Design secrets and key management for an AI system so that no secret lives in a prompt or artifact.
  6. Design secure fail states and safe degradation, including non-AI fallbacks.
  7. Assess a deployment against secure-by-design principles and produce architecture assurance evidence.

Module 14: Securing AI Pipelines and MLOps Environments

What the module covers. The pipeline that turns source, data, and a base model into a deployed system is itself an attack surface, and one that is often less defended than the application it produces. A compromise here is high-leverage: a poisoned dataset, a trojaned pretrained model, a malicious build dependency, or a tampered artifact propagates to every consumer of the resulting model, and does so with the appearance of a legitimate release. This chapter applies software supply-chain security (the Secure Software Development Framework, SLSA, signing and attestation, reproducibility) to the AI pipeline, adds the AI-specific concerns of dataset versioning, model lineage, checkpoint integrity, and model serialization risk, and shows how evaluation gates, an artifact registry, and infrastructure as code combine into a pipeline whose outputs can be trusted because their provenance can be verified.

Learning objectives (7)
  1. Map the AI pipeline from source to serving and identify the integrity boundary at each hand-off.
  2. Explain artifact signing, provenance attestation, and the SLSA build levels, and apply them to models and datasets.
  3. Design dataset versioning and model lineage so that any deployed model can be traced to its inputs.
  4. Analyze pipeline attack paths and map them to MITRE ATLAS supply-chain and development-time techniques.
  5. Assess model serialization formats for code-execution risk and choose safe alternatives.
  6. Design evaluation gates and a registry that cannot be bypassed and that promote only verified artifacts.
  7. Specify reproducibility, secrets, ephemeral-environment, and infrastructure-as-code controls with evidence.

Module 15: Secure Model Serving, APIs, Identity, and Access Control

What the module covers. Between the pipeline that produces a model and the architecture that surrounds it lies the serving layer: the endpoints that expose the model, the identities under which requests run, and the authorization decisions that decide what each request may retrieve and do. This chapter concentrates on identity and access control for AI systems, because that is where the confused-deputy problem of Chapters 3 and 12 is solved or lost. It covers model endpoint security and API hardening, authentication of users and workloads, the propagation of the end user's identity through the application to the model, tools, and enterprise resources, authorization before retrieval and before action, service accounts and least privilege, tenant separation, and the rate limiting and query controls that bound abuse and extraction. The central prescription is simple and hard: the system should act with the user's authority, not its own.

Learning objectives (7)
  1. Secure a model-serving endpoint and apply the OWASP API Security Top 10 to AI APIs.
  2. Distinguish user identity, workload identity, and service accounts, and explain when each is appropriate.
  3. Design end-user identity propagation using OAuth 2.0, OpenID Connect, and token exchange / on-behalf-of flows.
  4. Enforce authorization before retrieval and before action under the user's identity.
  5. Design tenant separation and least-privilege service accounts for a multitenant AI system.
  6. Apply rate limiting, quotas, and query controls to bound abuse, extraction, and unbounded consumption.
  7. Assess an AI system's identity and access design and produce authorization assurance evidence.

Module 16: AI Monitoring, Logging, Detection, and Incident Response

What the module covers. When an AI incident occurs, the first question is "what happened?" and the second is "which users, data, and actions were affected?" Neither can be answered without telemetry designed in advance. This chapter defines the AI telemetry model (prompts, context, model and prompt versions, retrieval, tool calls, identities, approvals, outputs, and outcomes) and shows how to correlate it into traces that reconstruct an incident. It covers privacy-aware logging, because that telemetry contains personal data; detection of the attacks of Part II from behavioral and content signals; and incident response for AI, mapping the functions of the NIST Cybersecurity Framework and SP 800-61r3 to concrete AI actions: disable a tool, revoke a token, roll back a model, remove a poisoned corpus, and preserve evidence. The chapter's premise is that AI systems are nondeterministic and manipulable, so detection is statistical and reconstruction depends entirely on what was captured.

Learning objectives (7)
  1. Specify the AI telemetry model and correlate it into per-request traces.
  2. Design privacy-aware logging that enables reconstruction without over-retaining personal data.
  3. Detect prompt injection, extraction, poisoning, abuse, and anomalous agent behavior from telemetry.
  4. Map incident response for AI to the NIST CSF 2.0 functions and SP 800-61r3.
  5. Define AI-specific containment, eradication, and recovery actions and their evidence.
  6. Reconstruct an AI incident from traces and produce a defensible timeline.
  7. Assess an AI system's observability and produce monitoring and IR readiness evidence.

Module 17: AI Supply Chain Risk and Third-Party Model Risk

What the module covers. Almost no organization builds its AI systems from scratch. They assemble them from pretrained models, public and purchased datasets, open-source packages, container images, third-party tools and MCP servers, and hosted model APIs. Every one of these is a dependency whose weaknesses the assembled system inherits: a backdoor in a base model, a memorized secret, a vulnerable library, a malicious serialization payload, a license problem, or a hidden instruction in a tool description. This chapter treats AI supply-chain risk as a first-class discipline, extending established software supply-chain security to the AI-specific artifacts of models, datasets, and tools. It covers provenance and signed artifacts, model serialization risk revisited, dependency and AI-bill-of-materials inventories, vendor due diligence, and the special problem of depending on a model provider whose behavior and security you cannot inspect. It complements Chapter 14, which secured the internal pipeline; this chapter secures what flows into it.

Learning objectives (7)
  1. Enumerate the components of the AI supply chain and the risks each carries.
  2. Explain how provenance, signing, and attestation establish trust in third-party artifacts and their limits.
  3. Assess model serialization and model-hub risks and select safe-handling controls.
  4. Construct an AI bill of materials using CycloneDX or SPDX and explain its uses.
  5. Design vendor due diligence for model, data, tool, and API providers.
  6. Analyze the risks of depending on a hosted model provider and specify contractual and technical controls.
  7. Assess an organization's AI supply-chain posture and produce inventory and provenance evidence.

Module 18: Testing, Evaluation, Red Teaming, and Assurance

What the module covers. Every preceding chapter ended with the same demand: prove that the control works and keep the evidence. This chapter is about how. It distinguishes validation, verification, and evaluation; explains security evaluation and adversarial testing for AI systems; sets out the limits of benchmarks and the discipline that keeps evaluations honest; describes red teaming and purple teaming for AI; defines the metrics that matter (attack success rate, severity, robust accuracy, true-positive rate at low false-positive rates) and the errors that flatter results; and shows how release gates and continuous evaluation turn a point-in-time test into ongoing assurance. The chapter draws together the assurance sections of Chapters 7-17 into a single method: an assurance argument that traces each security claim through a control and a test to evidence and a stated residual risk, run at a gate and maintained continuously.

Learning objectives (7)
  1. Distinguish validation, verification, and evaluation, and place each in the lifecycle.
  2. Design a security evaluation with attack cases, datasets, metrics, and thresholds tied to security claims.
  3. Explain the limits of benchmarks (overfitting, contamination, nonadaptive evaluation) and avoid them.
  4. Plan and conduct an AI red-team exercise and a purple-team exercise using ATLAS and OWASP.
  5. Choose metrics appropriate to each attack class and interpret them correctly.
  6. Design a nonbypassable release gate and a continuous-evaluation regime.
  7. Assemble an assurance argument from claims, controls, tests, evidence, and residual risk.

Part IV: Governance, Risk, and Workforce Readiness

Module 19: NIST AI RMF and AI Risk Management

What the module covers. The technical controls of Parts II and III do not organize themselves. Someone must decide which risks matter, assign ownership, require the evidence, and integrate AI risk into the enterprise's existing risk management. The NIST AI Risk Management Framework (AI RMF) is a widely used structure for doing so, and this chapter explains it as an operating model rather than a document to cite. It covers the trustworthy-AI characteristics the framework aims to preserve, its four functions (Govern, Map, Measure, Manage) and how they connect to the threat modeling, evaluation, and controls of earlier chapters; profiles, including the Generative AI Profile; AI actors and risk tolerances; and how AI risk integrates with enterprise risk management. The chapter frames the AI RMF as the connective tissue between the engineering of Parts II-III and the governance and compliance of Chapters 21-22.

Learning objectives (7)
  1. Describe the trustworthy-AI characteristics and distinguish security from the others.
  2. Explain the Govern, Map, Measure, and Manage functions and map earlier chapters onto them.
  3. Apply the framework to a system: establish context, identify risks, measure them, and manage them.
  4. Explain the role of profiles and summarize the Generative AI Profile's added risk categories.
  5. Identify AI actors and relate risk tolerances to control decisions.
  6. Integrate AI risk management with enterprise risk management and the risk register (Chapter 22).
  7. Assess the current status of the framework and its updates and cite it accurately.

Module 20: MITRE ATLAS, OWASP GenAI, and AI Security Frameworks

What the module covers. The previous chapters have drawn on several AI security frameworks (the NIST adversarial ML taxonomy, MITRE ATLAS, the OWASP LLM and Agentic Top 10s, the NIST CSF and Privacy Framework), often in the same paragraph. This chapter steps back to explain what each framework is for, what it covers and does not, and how to combine them without pretending they are equivalent or that one subsumes another. Each answers a different question at a different layer: a shared vocabulary of attacks, a catalog of adversary techniques, a prioritized list of application risks, an enterprise control structure. Using them well means routing each question to the framework built for it and mapping between them consciously, which is the purpose of the crosswalk in Appendix C.

Learning objectives (6)
  1. State the purpose, scope, and structure of MITRE ATLAS, the OWASP LLM and Agentic Top 10s, and the NIST AML taxonomy.
  2. Explain how each relates to the NIST AI RMF and to enterprise frameworks.
  3. Route a given security question to the framework designed to answer it.
  4. Combine the frameworks for threat modeling, evaluation, and control selection without conflating them.
  5. Map between frameworks conceptually and recognize where mappings are many-to- many.
  6. Assess the currency of each framework and cite it accurately.

Module 21: AI Governance, Policy, Compliance, and the EU AI Act

What the module covers. Governance is the machinery that decides what an organization will and will not do with AI, assigns accountability, and verifies that the technical work of Parts II-III actually happens. This chapter describes an AI governance operating model (board accountability, a cross-functional committee, an AI inventory, policies, risk classification, assurance, and audit) and connects it to the system-level work of the rest of the book. It then treats the standards and regulation that the model must satisfy: ISO/IEC 42001 (a certifiable AI management system standard), ISO/IEC 23894 and 27090, and the EU AI Act's risk-based structure, including the 2026 Digital Omnibus amendments to its timelines. The chapter is careful to distinguish technical guidance from legal obligation and states clearly that it is orientation, not legal advice; specific obligations and dates must be verified with qualified counsel for the relevant jurisdiction.

Learning objectives (7)
  1. Describe an AI governance operating model and the role of each component.
  2. Explain the purpose of an AI inventory, policies, and risk classification and how they drive controls.
  3. Summarize ISO/IEC 42001 and its relationship to ISO 27001 and the NIST AI RMF.
  4. Explain the EU AI Act's risk-based tiers and the current status of its timelines.
  5. Distinguish technical guidance from legal requirement and state jurisdiction and date context for regulatory claims.
  6. Relate governance and compliance to the security engineering of Parts II-III.
  7. Assess an organization's AI governance maturity and produce governance evidence.

Module 22: Building AI Risk Registers and Control Mappings

What the module covers. A risk register is where the whole book comes together in a form the organization can act on. It records each risk as a statement, rates it, links it to the controls that treat it and the evidence that the controls work, states the residual risk and who accepts it, and crosswalks it to the frameworks and obligations of Chapters 19-21. This chapter sets out the anatomy of an AI risk-register entry (risk statement, inherent risk, controls, evidence, residual risk, ownership, treatment, and review) using the running Walia Ibex example, and shows how crosswalks let one register serve security, privacy, and compliance at once. It turns the assurance argument of Chapter 18 and the governance of Chapter 21 into the living document that a security team maintains and an auditor reads. Appendix B provides the reusable template.

Learning objectives (7)
  1. Write an AI risk statement with asset, threat, vulnerability, and impact.
  2. Rate inherent and residual risk using likelihood and impact, and explain the difference.
  3. Link each risk to controls with owners and to evidence that the controls work.
  4. Record a treatment decision (accept, mitigate, transfer, avoid) with an accountable owner.
  5. Crosswalk a register entry to the NIST AI RMF, CSF, Privacy Framework, ATLAS/OWASP, and ISO/EU obligations.
  6. Handle exceptions and risk acceptance with accountability and review.
  7. Maintain the register as a living document integrated with enterprise risk management.

Module 23: SecureAI Workforce Roles and Career Pathways

What the module covers. AI security is a team discipline: no single role holds the threat modeling, adversarial ML, MLOps, identity, monitoring, evaluation, governance, and legal knowledge the preceding chapters require. This chapter describes the roles that make up an AI security capability, the skills and responsibilities of each, how they map to the work of the book, and how they combine into an interdisciplinary team. It relates the roles to the NIST NICE Workforce Framework for Cybersecurity and to the AI-actor concept of the NIST AI RMF, and it sketches career pathways into and across these roles. It is written for readers deciding how to staff an AI security function or where to take their own careers, and it is deliberately non-normative about titles, which vary across organizations.

Learning objectives (6)
  1. Describe the principal AI security roles and their responsibilities.
  2. Map each role to the chapters and activities of this book.
  3. Relate the roles to the NICE Framework and the AI RMF's AI actors.
  4. Design an interdisciplinary AI security team for an organization of a given size and risk profile.
  5. Identify the skills and pathways into and across the roles.
  6. Explain why AI security requires collaboration across security, ML, cloud, privacy, and governance.

Module 24: Capstone: Threat Model, Architecture Review, Red Team, and Assurance Report

What the module covers. The capstone integrates the entire book by applying it to a single AI system, from system context to an executive assurance report. It is a structured project with thirteen deliverables that together exercise the mechanisms of Part I, the threat analysis of Part II, the engineering and testing of Part III, and the governance and risk of Part IV. This chapter specifies the deliverables, the workflow that produces them, the assessment rubric, and the ethical constraints, and it walks through the running Walia Ibex example as a worked reference. It is written for students completing a course, for practitioners conducting a real assessment, and for instructors designing the assessment; the same structure serves all three.

Learning objectives (7)
  1. Integrate the book's methods into a single, coherent security assessment of an AI system.
  2. Produce the thirteen capstone deliverables to a professional standard.
  3. Trace each deliverable to the chapters and mechanisms it draws on.
  4. Distinguish deterministic from probabilistic controls and reason about containment in the design.
  5. Conduct a controlled red-team exercise and rate findings by severity.
  6. Assess and communicate residual risk to a nontechnical decision-maker.
  7. Apply the ethical constraints of authorized testing throughout.