You asked the model for JSON. It returned JSON… with a markdown code fence around it. Or an extra field. Or a string where you expected a number. Your parser crashes. Sound familiar?

The 5 most common parsing failures

1. Markdown wrapping

Model returns: ```json\n{"name": "John"}\n```
You expected:  {"name": "John"}

Fix: Strip markdown fences before parsing. Or use structured outputs which never add wrapping.

2. Schema drift

You expected: {"name": "John", "age": 30}
Model returns: {"full_name": "John Doe", "years_old": 30}

The model invents its own field names. Happens more with smaller models and vague prompts.

Fix: Structured outputs enforce your exact schema. Or validate with Zod/Pydantic and retry on failure.

3. Type mismatches

You expected: {"count": 5}
Model returns: {"count": "five"}

Fix: Structured outputs enforce types. Without them, cast and validate explicitly.

4. Extra text before/after JSON

Model returns: "Here's the data:\n{"name": "John"}\nHope that helps!"

Fix: Extract JSON with regex: /\{[\s\S]*\}/. Or use structured outputs.

5. Hallucinated fields

You expected: {"name": "John"}
Model returns: {"name": "John", "confidence": 0.95, "source": "inferred"}

The model adds fields you didn’t ask for. Usually harmless but can break strict parsers.

Fix: Use additionalProperties: false in your JSON schema. Or strip unknown fields after parsing.

The real fix: structured outputs

All five problems disappear with structured outputs. The model is constrained to your exact schema at the token generation level — it literally cannot produce invalid output.

response = client.chat.completions.create(
    model="gpt-5.4",
    response_format={"type": "json_schema", "json_schema": {"schema": your_schema}},
    messages=[...]
)
# Always valid. Always matches schema. No parsing surprises.

When you can’t use structured outputs

For models that don’t support structured outputs (some open-source models via Ollama):

1. Ask for JSON explicitly in the prompt
2. Parse with try/catch
3. On failure, retry with the error message
4. Validate with Zod/Pydantic
5. Max 3 retries, then fallback

See our structured outputs guide and schema-first design guide.

Related: Structured Outputs Explained · JSON Mode vs Structured Outputs · Schema-First AI App Design · When Not To Use Ai Agents