Apr 29, 2026 · 4 min read

Last updated on Jun 12, 2026

AI Security Checklist for Startups — Protect Your LLM Application

You’re building with AI. Your app calls LLM APIs, maybe uses MCP for tool integration, maybe has a RAG pipeline. Here’s the security checklist before you go to production.

Input security

✅ Sanitize user input before it reaches the LLM Strip or flag common prompt injection patterns. Not foolproof, but catches obvious attacks.

✅ Validate all structured inputs If using tool calling or MCP, validate every parameter with Zod/Pydantic. Never trust raw LLM output as function arguments.

✅ Limit input length Set maximum token limits per request. Prevents abuse and controls costs.

Output security

✅ Filter sensitive data from responses Check LLM output for API keys, internal URLs, system prompts, or PII before returning to users.

✅ Use structured outputs where possible Constrained output is harder to exploit than free-form text.

✅ Don’t expose raw model errors to users Catch exceptions and return generic error messages. Model error details can leak information.

Data security

✅ Never put secrets in system prompts Assume system prompts can be extracted via prompt injection. Use environment variables and server-side logic instead.

✅ Know where your data goes Every API call sends data to a third party. See our data privacy guide for what each provider does with your data.

✅ Use API access, not consumer subscriptions Consumer plans (ChatGPT Plus, Claude Pro) may use your data for training. API access with a DPA doesn’t. See our GDPR guide.

✅ Redact PII before sending to LLMs If your prompt contains user data, strip names, emails, and addresses before the API call. Re-insert after.

✅ Remove founder/team personal data from broker sites Personal data on broker sites can be used for social-engineering attacks against your startup. Review major broker listings and document removal requests for founders and key staff.

Infrastructure security

✅ Rotate API keys regularly Set a reminder to rotate keys quarterly. Use a secrets manager, not .env files in repos.

✅ Set spending limits Every provider offers budget caps. Set them. See our cost monitoring guide.

✅ Sandbox MCP servers Run MCP servers in containers with limited network access. A compromised server shouldn’t access your entire infrastructure.

✅ Use least privilege for tool access Database MCP servers should be read-only. File system access should be limited to specific directories. See our MCP Security Checklist.

Model supply chain

✅ Audit open-source models before using them Models from HuggingFace can contain malicious code in custom layers. Use models from known organizations (Mistral, Google, Z.ai).

✅ Pin model versions Don’t use latest in production. Pin to specific versions so updates don’t silently change behavior.

✅ Verify model checksums When downloading model weights, verify SHA256 checksums match the published values.

Monitoring

✅ Log all LLM interactions Every prompt and response should be logged for audit. See our observability guide.

✅ Alert on anomalies Unusual patterns (high token usage, repeated failures, off-hours access) may indicate an attack.

✅ Have an incident response plan What do you do if your system prompt leaks? If a user extracts customer data? Have a plan before it happens.

The priority order

If you can only do 5 things:

Set spending limits (prevents financial damage)
Don’t put secrets in prompts (prevents data leaks)
Use API access with DPA, not consumer plans (prevents compliance issues)
Validate tool calling inputs (prevents injection via tools)
Log everything (enables incident response)

Security incident response plan

When (not if) something goes wrong with your AI system:

Immediate (first 15 minutes)

Assess scope — what data was exposed? Which users affected?
Contain — disable the AI feature, revoke compromised API keys
Preserve evidence — don’t delete logs, take screenshots

Short-term (first 24 hours)

Investigate — review logs to understand what happened
Notify — inform affected users if personal data was exposed (GDPR requires notification within 72 hours)
Fix — patch the vulnerability, add it to your red team test suite

Post-incident

Post-mortem — document what happened, why, and how to prevent it
Update checklist — add new checks based on the incident
Test the fix — verify the vulnerability is actually closed

Common AI security mistakes

Mistake	Impact	Fix
API key in git	Key stolen in seconds	Use password manager, env vars
No rate limiting	$10K bill from abuse	Add per-user rate limits
System prompt in response	Attackers learn your logic	Filter system prompt from output
PII in prompts	Data sent to third party	Redact before sending to API
No input validation	Prompt injection	Sanitize and validate all user input
Consumer plan for business	No DPA, data used for training	Use API/enterprise plans

Quarterly security review

Set a calendar reminder to review these quarterly:

All API keys rotated
Spending limits still appropriate
Provider terms of service checked for changes
Team trained on latest AI security best practices
Risk assessment updated