A TLS handshake negotiates protocol settings, validates the server identity, and establishes encrypted session keys before an AI request is sent. A failure therefore occurs before API authentication, model execution, or MCP tool authorization.

Separate the failure layers

Common classes are:

Symptom	Likely boundary
Certificate verification error	chain, hostname, expiry, or CA trust
Protocol/cipher alert	incompatible TLS policy
Connection closes during handshake	proxy, load balancer, mTLS, or firewall
Works directly but not through gateway	SNI, upstream trust, or proxy configuration
Only one container fails	image CA bundle or runtime trust store

Inspect without exposing credentials:

openssl s_client -connect gateway.example.com:443 -servername gateway.example.com -showcerts
curl -v https://gateway.example.com/health

The -servername value matters because gateways often host multiple certificates behind one address.

Validate both sides of a gateway

An AI gateway has at least two TLS relationships: client-to-gateway and gateway-to-provider or internal service. Test them independently. A correct public certificate says nothing about the gateway’s trust of an internal model server.

For streaming responses, also verify that proxies do not terminate long-lived SSE connections or buffer data unexpectedly. TLS may succeed while the stream later resets; treat that as an operations problem rather than weakening TLS.

mTLS and internal AI services

Mutual TLS requires the client to present an accepted certificate. Confirm:

• client certificate and private key match;
• issuing CA is trusted by the server;
• certificate identity maps to the intended service;
• validity and revocation policy are current;
• rotation leaves an overlap window;
• private keys are injected through the secrets system.

mTLS authenticates a workload, but it does not replace application authorization. MCP tools and agents still need scoped permissions. See MCP authentication and OAuth for AI agents.

Never solve this by disabling verification

Global verification bypasses can expose model-provider keys, prompts, outputs, and tool credentials. Correct the certificate chain, trust store, hostname, clock, protocol policy, or proxy instead. Use TLS certificate chain troubleshooting for issuer errors and expired certificate recovery for validity failures.

Release and monitoring checklist

1. Test client, gateway, and upstream TLS separately.
2. Test from the actual container and network.
3. Verify SNI, hostname, chain, and trust store.
4. Exercise mTLS identity and rotation where used.
5. Run a real authenticated request with non-production credentials.
6. Test streaming, timeouts, and rollback.
7. Alert on handshake failures by service and certificate.

The broader controls belong in AI Security; deployment belongs in AI Deployment & Hosting, and runtime symptoms belong in AI Operations.