AI Audit Checklist: What Auditors Will Ask For
Short answer: an AI audit asks one thing: can you prove your AI systems are governed, tested and monitored the way your policy says? This checklist is the evidence you will be asked for.
1. Inventory and ownership
- A complete list of AI systems you build, buy and use (including AI inside vendor tools)
- A named, accountable owner for each system
- A risk rating for each use case
2. Policy and risk criteria
- A written AI policy approved by leadership
- Risk criteria: what counts as high risk, and who signs off
- Mapped to your framework (e.g. NIST AI RMF Govern function, ISO/IEC 42001)
3. Data
- Data sources documented for each system
- Lineage: where data came from and how it was transformed
- Data quality controls (completeness, accuracy, timeliness)
- Access and usage rights, including privacy obligations
4. Testing and evaluation
- Pre-deployment testing results
- Known limitations and failure modes written down
- Bias and fairness checks where people are affected
- Human review points for high-impact decisions
5. Monitoring
- Output monitoring and drift detection in production
- Defined fallback when confidence is low
- Cost and usage limits for autonomous agents
6. Evidence and change control
- Logs showing how an answer was produced (inputs, prompts, versions)
- Change history for models, prompts and data
- Incident records and what was done about them
The audit debt trap
AI makes building cheap and verification expensive. Every system you add without this evidence adds to your audit debt. Read more: Audit Debt: the hidden cost of AI automation.
FAQ
What is an AI audit?
An AI audit checks whether an organization's AI systems are governed, documented, tested and monitored in line with its policies and the frameworks it follows, such as ISO/IEC 42001 or the NIST AI RMF.
What evidence do you need for an AI audit?
An AI system inventory, named owners, an AI policy and risk criteria, data lineage and quality records, test and evaluation results, monitoring logs, incident records, and change history showing how outputs were produced.
How do I prepare for an AI audit?
Start with a complete inventory, assign ownership, then close gaps system by system, starting with the highest-risk use cases. Keep evidence as you go instead of rebuilding it before the audit.