Model output that looks like JSON is not necessarily valid or safe to execute. It may be truncated, wrapped in Markdown, shaped differently from the schema or replaced by an HTML error page from a gateway.

The reliable approach is to distinguish transport, syntax, schema and semantic failures. This guide belongs with AI API design and TypeScript for AI applications.

Diagnose before parsing

Check the HTTP response and content type before calling JSON.parse:

const response = await fetch(url, options);
const raw = await response.text();

if (!response.ok) {
  throw new Error(`Upstream ${response.status}: ${raw.slice(0, 200)}`);
}

if (!response.headers.get("content-type")?.includes("application/json")) {
  throw new Error("Expected JSON but received another content type");
}

An unexpected < often means a proxy returned HTML. Retrying JSON parsing cannot fix that.

Parse and validate separately

Valid JSON can still violate the application contract:

import { z } from "zod";

const DecisionSchema = z.object({
  action: z.enum(["approve", "reject", "review"]),
  reason: z.string().min(1),
  confidence: z.number().min(0).max(1),
});

let value: unknown;
try {
  value = JSON.parse(raw);
} catch {
  throw new Error("Model response was not valid JSON");
}

const result = DecisionSchema.safeParse(value);
if (!result.success) {
  throw new Error("Model response did not match the decision schema");
}

Never replace validation with a type assertion. Runtime model output is untrusted input.

Prefer provider-supported structured outputs

Where available, define a response schema through the model API rather than asking for “JSON only” in prose. Structured-output modes reduce syntax errors, but the application must still validate:

• providers can reject unsupported schemas;
• model or SDK versions can change behaviour;
• a valid shape can contain unsafe or nonsensical values;
• transport truncation can occur outside the model.

Keep schemas small. Deep unions and ambiguous optional fields make both generation and recovery harder.

Tool calls require authorization

A syntactically correct tool call is not permission to execute it. Validate arguments, identify the requesting user or agent and apply business authorization before side effects.

const DeleteArgs = z.object({ projectId: z.string().uuid() });
const args = DeleteArgs.parse(toolCall.arguments);
await requireProjectAdmin(user.id, args.projectId);

Use idempotency for retryable tool calls. A second valid request must not create a second charge, cancellation or destructive action.

Truncated JSON and streaming

Do not parse a streaming response chunk-by-chunk as though each chunk were a JSON document. Use the provider’s event protocol and wait for a completed structured event.

If a response ends early, record:

• provider request ID;
• finish or stop reason;
• bytes or tokens received;
• timeout/reset at each proxy hop;
• model and schema version.

Avoid logging the complete response when it may contain personal or proprietary data.

Retry carefully

Retry only failures likely to change:

• transient gateway error;
• rate limit with an explicit retry delay;
• truncated network response;
• model schema failure where a bounded repair attempt is allowed.

Do not retry indefinitely. Use a maximum attempt count, exponential backoff and a fallback path. For high-impact decisions, return a review state instead of inventing a default.

Repair strategies

In order of preference:

1. Retry through a structured-output API with the same schema.
2. Ask a model to repair the invalid value without adding new facts.
3. Route to a more reliable model or deterministic parser.
4. Ask the user to correct input or review the result.

Regex extraction of the first {...} block is fragile around nested objects and braces inside strings. Use it only for low-risk recovery with validation afterward.

Stable error contract

Expose a controlled error to clients:

{
  "error": {
    "code": "MODEL_OUTPUT_INVALID",
    "retryable": true,
    "requestId": "req_123"
  }
}

Do not return raw provider output, stack traces or prompts. Internally, retain enough redacted context to reproduce the failure.

Invalid model JSON is a normal boundary failure, not an exceptional mystery. A production AI application detects it, prevents unsafe execution and degrades in a way users can understand.