Evaluating an AI Tool's Security Posture
Before adding an AI tool to your stack, evaluate its security. This guide covers SOC 2, encryption, data residency, retention, access controls, penetration testing, incident response, and a practical checklist. Red flags to watch for.
SOC 2 Certification
What it is — Audited controls for security, availability, processing integrity, confidentiality, and privacy. Type I (design) and Type II (operating effectiveness).
Why it matters — Indicates the vendor has documented and tested security practices. Often required by enterprise customers.
Check — Is SOC 2 available? Type I or II? When was it last audited? Can you get a summary or report under NDA?
Encryption
At rest — Data stored encrypted. AES-256 common. Key management matters.
In transit — TLS 1.2 or 1.3. No unencrypted HTTP for sensitive data.
Check — Encryption at rest and in transit. Key management (who holds keys, rotation). Some tools offer bring-your-own-key (BYOK).
Data Residency
What it is — Where data is stored. Geographic restrictions (EU data in EU, etc.) for compliance.
Check — Can you choose region? Where is data stored by default? Do they support your residency requirements?
Retention Policies
What it is — How long data is kept. Affects compliance and risk.
Check — Retention period. Can you set shorter retention? Auto-deletion? Export before deletion?
Access Controls
What it is — Who can access data. Authentication, authorization, role-based access.
Check — MFA? SSO? Role-based permissions? Audit logs for access? Least privilege?
Penetration Testing
What it is — Simulated attacks to find vulnerabilities. Third-party testing is more credible.
Check — Do they conduct pen tests? How often? Can you see a summary (redacted)? Do they remediate findings?
Incident Response
What it is — How they handle breaches and security incidents.
Check — Incident response plan? Breach notification process? SLA for notification? Do they have cyber insurance?
Practical Security Checklist
- SOC 2 (Type II preferred)
- Encryption at rest and in transit
- Data residency options
- Retention policy and controls
- MFA and SSO
- Access audit logs
- Pen testing (regular, third-party)
- Incident response and breach notification
- Security documentation (available on request)
Red Flags
- No SOC 2 or equivalent
- No encryption details
- No data residency choice
- No retention or deletion controls
- No MFA
- No breach notification commitment
- Vague or absent security documentation
The Bottom Line
Evaluate security before adopting AI tools. Use the checklist. Prefer SOC 2, encryption, residency options, and clear incident response. Red flags should prompt deeper scrutiny or alternative vendors.