Agent Security Audit Kit

End-to-end security audit for AI agents: a 5-phase methodology, 35 controls mapped to the OWASP Top 10 for LLM Applications 2026 and NIST AI RMF functions, evidence templates and register, a scoring script, a read-only MCP config scanner and a report template.

Doc / CSV / Python scripts (ZIP + PDF)End-to-end security audit for AI agents: a 5-phase methodology, 35 controls mapped to the OWASP Top 10 for LLM Applications 2026 and NIST AI RMF functions, evidence templates and register, a scoring script, a read-only MCP config scanner and a report template.

What's inside

Verify it works

  1. python3 score_audit.py --selftest prints selftest ok. It checks that all 35 controls have unique ids, that every OWASP 2026 category has at least one mapped control, that the NIST function labels are valid, and that scoring and the critical-cap rule behave correctly.
  2. python3 scan_mcp_config.py --selftest prints selftest ok. It runs the scanner against built-in risky and safe sample configs and confirms no secret values are printed.
  3. Dry run: copy controls.csv, mark a few controls pass/partial/fail with an evidence_ref, then run python3 score_audit.py your_copy.csv. You get findings.md with domain, OWASP and NIST tables and a sorted gap list. Blank results are listed as CHECK lines.
  4. Point scan_mcp_config.py at a real MCP client config. It only reads the file, and never prints secret values.
  5. Open 01_Audit_Methodology.md and plan your scope; the report skeleton is 03_Audit_Report_Template.md. Python 3.9+, no installs.

Preview

First part of the main document, shown as-is:

Agent Security Audit: Methodology

A repeatable, evidence-based audit of AI agents in production. One person can run it for a small fleet in about [3–5] days; the same steps scale to a team audit.

What you get at the end: a scored control assessment, an evidence register an outside auditor can follow, a prioritized gap list and a signed report.

Frameworks referenced. Each control in controls.csv is mapped to the risk categories of the OWASP Top 10 for LLM Applications 2026 (https://genai.owasp.org/resource/owasp-genai-llm-top-10-2026/) and to the four functions of the NIST AI Risk Management Framework, GOVERN, MAP, MEASURE and MANAGE (https://www.nist.gov/itl/ai-risk-management-framework). These mappings are our own judgement, to help you show coverage. They are not an official crosswalk, and passing this audit is not a certification.

Roles

Role Does
Lead auditor Plans, tests controls, scores, writes the report. Independent of the agent team where possible.
Agent owner(s) Provides access, demos, evidence; answers interviews.
Security / GRC sponsor Approves scope, accepts risk, signs off.

Phase 1: Scope (half day)

  1. List the agents in scope, with environment, owner and risk tier. Use GOV-01 as the source; if no inventory exists, building one is finding #1.
  2. Pick depth per agent. Full for agents with R2/R3 tools (external messages, money, deletion, permissions) or regulated data; light (Governance, Tools, Monitoring domains only) for read-only internal agents.
  3. Agree on the rules of engagement: staging for active tests, no real customer data in test inputs, a named contact for stopping a test.
  4. Record all of this in section 1 of 03_Audit_Report_Template.md.

Phase 2: Discover (1 day)

← All kits