Your team wants to use an AI API in production. Procurement needs a vendor assessment. Here are the 25 questions that actually matter, organized by category.
Security (5 questions)
1. Where is data processed? US, EU, or other? Does the vendor offer region selection? Critical for GDPR and data sovereignty.
2. Is data used for training? API access typically isn’t. Consumer plans often are. Get it in writing. See our data privacy guide.
3. What’s the data retention policy? How long are prompts and responses stored? Can you set custom retention periods?
4. Is there a DPA (Data Processing Agreement)? Required under GDPR. Most enterprise API plans include one. Consumer plans don’t.
5. What security certifications does the vendor hold? SOC 2 Type II is the minimum. ISO 27001 is better. Check if certifications cover the AI-specific infrastructure.
Compliance (4 questions)
6. How does the vendor classify AI risk under the EU AI Act? If your use case is high-risk, the vendor needs to support your compliance obligations.
7. Can the vendor provide model cards and technical documentation? Required for high-risk AI systems under the EU AI Act.
8. Does the vendor support audit trails? Can you export logs of all AI interactions for regulatory review?
9. How does the vendor handle bias and fairness? What testing has been done? Are results published?
Technical (5 questions)
10. What’s the API compatibility? OpenAI-compatible? Anthropic-compatible? Proprietary? This affects your ability to switch vendors. OpenRouter can abstract this.
11. What are the rate limits and quotas? Requests per minute, tokens per day, concurrent connections. Will they scale with your usage?
12. What’s the uptime SLA? 99.9%? 99.95%? What’s the compensation for downtime?
13. Does the vendor support structured outputs? Critical for production applications that need reliable JSON responses.
14. Is there MCP support? For tool integration, MCP compatibility means your integrations work across vendors.
Pricing (4 questions)
15. What’s the per-token pricing? Input vs output rates. Compare across providers — prices vary 50x. See our pricing comparison.
16. Are there volume discounts? Most providers offer committed-use discounts at scale.
17. Is there a free tier or trial? Test before committing. See our free AI APIs guide.
18. What are the hidden costs? Prompt caching fees, fine-tuning costs, storage fees, support tiers.
Operational (4 questions)
19. What’s the model update policy? Will the vendor change models without notice? Can you pin versions?
20. What support is available? Email only? Dedicated account manager? Response time SLAs?
21. Is there a status page and incident communication? How will you know when the service is degraded?
22. Can you self-host as a fallback? Open-weight models (Mistral, GLM, DeepSeek) give you an exit option.
Exit strategy (3 questions)
23. What happens to your data when you leave? Deletion timeline, export options, certification of deletion.
24. How portable is your integration? If you use OpenAI-compatible APIs, switching to another provider is a config change. Proprietary APIs create lock-in.
25. Can you run the same model elsewhere? Open-weight models can be self-hosted. Proprietary models can’t. This is your ultimate exit strategy.
The quick assessment
If you need a fast evaluation, focus on these 5:
- Is there a DPA? (compliance)
- Where is data processed? (sovereignty)
- Is data used for training? (privacy)
- What’s the per-token cost? (budget)
- Is the API OpenAI-compatible? (portability)
These five questions eliminate 80% of unsuitable vendors in 10 minutes.
Red flags during vendor evaluation
Watch out for these warning signs:
- “We don’t use your data for training” but it’s not in the terms of service — get it in writing
- No DPA available — if they process personal data and can’t provide a DPA, they’re not GDPR-ready
- Vague uptime SLA — “we aim for 99.9%” is not a commitment. Look for contractual SLAs with compensation
- No model versioning — if they can change the model without notice, your app’s behavior can change overnight
- Proprietary API only — no OpenAI-compatible endpoint means maximum vendor lock-in
- No status page — if they don’t have a public status page, they’re not transparent about reliability
Negotiation tips
Once you’ve shortlisted vendors:
- Ask for a pilot — most enterprise AI vendors offer 30-90 day pilots with dedicated support
- Negotiate on volume — committed-use discounts of 20-40% are common above $1K/month
- Get exit terms in writing — data deletion timeline, export format, transition period
- Ask about roadmap — will they support the features you need in 6-12 months?
- Reference check — ask for customers in your industry and talk to them
Building your evaluation scorecard
Create a simple spreadsheet:
| Criteria | Weight | Vendor A | Vendor B | Vendor C |
|---|---|---|---|---|
| Data privacy | 25% | 4/5 | 5/5 | 3/5 |
| API compatibility | 20% | 5/5 | 3/5 | 4/5 |
| Pricing | 20% | 3/5 | 4/5 | 5/5 |
| Uptime SLA | 15% | 4/5 | 4/5 | 3/5 |
| Exit strategy | 10% | 3/5 | 5/5 | 2/5 |
| Support quality | 10% | 4/5 | 3/5 | 4/5 |
| Weighted total | 3.85 | 4.10 | 3.55 |
Adjust weights based on what matters most to your organization. For EU companies, data privacy should be weighted highest.
Related: AI and GDPR · Which AI APIs Are GDPR Compliant? · AI Governance for Startups · AI Coding Tools Pricing 2026 · AI Risk Assessment Template