Short answer: before you buy an AI or agent product, ask about five areas: how your data is used and retained, the vendor's security controls, AI-specific risks such as prompt injection and autonomous actions, incident handling and continuity, and regulatory fit. Score the answers, and treat vague answers on training and retention as red flags.
The five areas, with sample questions
A. Data use and retention
- Is customer data used to train or fine-tune any model, by default or ever? What is the default?
- How long are prompts, outputs, logs and backups kept? Can we set zero or shorter retention?
- Which subprocessors, including model providers, receive our data, and in which regions?
B. Security controls
- Which third-party attestations (for example SOC 2 Type II or ISO 27001) are current, with what scope and date?
- Do you support SSO/SAML, SCIM, role-based access and admin audit logs?
- How is tenant isolation enforced?
C. AI-specific risks
- How do you defend against prompt injection from documents, web content or emails the system processes?
- Can the system take actions (send, purchase, modify records)? What human-approval controls exist?
- How are model or prompt changes tested and announced to customers?
D. Incidents and continuity
- How fast will you notify us of an incident affecting our data?
- What happens to our data and workflows if a model provider changes or fails?
E. Regulatory fit
- Ask only what applies to you, for example healthcare, financial or regional data-protection requirements.
How to use the answers
Score each answer (for example 0 = no answer, 1 = partial, 2 = clear and evidenced), weight the areas that matter most for your data, and follow up on anything scored 0. A questionnaire is not a contract: your counsel and security team set the final requirements.
The AI Vendor Security & Data-Handling Questionnaire ($29) has all 24 questions across these five areas, with a reviewer scoring sheet.
FAQ
What should I ask an AI vendor about training on my data?
Whether customer data is used to train or fine-tune any model by default or ever, how that is controlled, and what the default setting is.
Is a SOC 2 report enough?
It covers general security controls, not AI-specific risks such as prompt injection, autonomous actions or model changes, so ask about those separately.
Is a questionnaire a contract?
No. Answers inform your review; your counsel and security team set the contract terms.