πŸ—οΈ AI Application Architecture
Β· 3 min read
Last updated on

Validate AI Model Responses with Zod and Structured Outputs


ZodError: expected string, received undefined

A model returned data that did not satisfy your runtime contract. Even when a provider supports structured outputs, validation remains necessary at your application boundary: schemas can drift, tool arguments can be incomplete, cached data may use an older version, and external input can bypass the model entirely.

Do not make required fields optional merely to silence the error. Decide whether the schema, prompt, provider contract or application assumption is wrong.

Validate unknown data at the boundary

import { z } from 'zod';

const Ticket = z.object({
  summary: z.string().min(1),
  priority: z.enum(['low', 'medium', 'high']),
  requiresHumanReview: z.boolean(),
});

const result = Ticket.safeParse(modelOutput);

if (!result.success) {
  console.error(result.error.issues);
  throw new Error('Model response failed validation');
}

const ticket = result.data;

Keep model output typed as unknown until parsing succeeds. Type assertions such as as Ticket do not validate runtime data.

JSON parsing and schema validation are different

Two separate failures can occur:

  1. The response is not valid JSON.
  2. It is valid JSON but does not match the schema.
let unknownValue: unknown;

try {
  unknownValue = JSON.parse(rawText);
} catch {
  throw new Error('Model returned malformed JSON');
}

const parsed = Ticket.safeParse(unknownValue);
if (!parsed.success) {
  throw new Error('Model returned JSON with the wrong shape');
}

Use the JSON parsing foundation for truncated responses, markdown fences and malformed syntax. Zod begins after syntactically valid data exists.

Structured outputs reduce failures, not trust boundaries

When an AI SDK or provider accepts a schema, send the narrowest schema that represents the task. Prefer enums, discriminated unions and bounded arrays over vague strings and open objects.

const ToolCall = z.discriminatedUnion('action', [
  z.object({
    action: z.literal('search'),
    query: z.string().min(1).max(500),
  }),
  z.object({
    action: z.literal('archive_project'),
    projectId: z.string().uuid(),
    reason: z.string().min(1),
  }),
]);

Schema validity does not prove that a tool call is authorized or correct. Validate permissions, current state and human-approval requirements after parsing and before execution.

Diagnose schema failures safely

Zod issues identify paths and constraints:

if (!result.success) {
  const safeIssues = result.error.issues.map(({ path, code, message }) => ({
    path: path.join('.'),
    code,
    message,
  }));
  logger.warn({ safeIssues }, 'AI response validation failed');
}

Do not log the complete model response automatically. It may contain personal data, proprietary context, secrets or harmful content.

Track failure rate by schema version, model, provider, prompt version and finish reason. That turns a recurring ZodError into useful evaluation evidence.

Safe retry policy

Retry only when another attempt can plausibly produce a valid result. A useful sequence is:

  1. Reject malformed or invalid data before side effects.
  2. Record a sanitized failure reason.
  3. Retry once with the same schema and a concise correction signal.
  4. Use a fallback model or non-AI path when appropriate.
  5. Escalate to human review for consequential actions.

Do not recursively feed full validation errors and previous output back to a model without limits. Retries consume money and can reproduce the same failure indefinitely.

type ParseResult<T> =
  | { ok: true; data: T }
  | { ok: false; reason: 'invalid_json' | 'schema_mismatch' };

Make failure explicit in the application contract rather than returning partially trusted objects.

Schema evolution

Stored model output, queued jobs and agent state may outlive a deployment. Version schemas when their meaning changes:

const V1 = z.object({
  schemaVersion: z.literal(1),
  answer: z.string(),
});

const V2 = z.object({
  schemaVersion: z.literal(2),
  answer: z.string(),
  citations: z.array(z.string().url()),
});

const StoredResult = z.discriminatedUnion('schemaVersion', [V1, V2]);

Migrate intentionally. Making new fields optional everywhere can hide incompatible data.

Test the contract

Maintain fixtures for valid output and expected failures:

import { expect, test } from 'vitest';

test('rejects an archive action without a project id', () => {
  const result = ToolCall.safeParse({
    action: 'archive_project',
    reason: 'inactive',
  });
  expect(result.success).toBe(false);
});

Then evaluate whether the model chooses the right action separately. Schema tests prove structure; evaluation proves behavior.

Connect this layer to structured outputs, the TypeScript foundation, AI API design, AI Application Architecture and AI Testing & Evaluation.

Primary documentation