Skip to Content

Structured Outputs and JSON Mode

When to use

Force the LLM to emit valid JSON or a typed schema — the foundation of safe tool calls and deterministic pipelines.

Analogy

Requiring a human to fill out a standard form rather than write an essay.

Data-flow diagram

   response_format = {"type":"json_object"}      -> emit valid JSON
   response_format = {"type":"json_schema",...}  -> emit schema-valid JSON

Deep explanation

JSON mode guarantees syntactic JSON: every output starts with { and ends with }. JSON-schema mode additionally enforces shape — object keys, value types, enums. Use it when downstream code parses the output. Schemas significantly reduce parse failures. Always validate at the boundary, even with schema mode.

Examples

Example 1

resp = openai.chat.completions.create(
    model='gpt-4o-mini', response_format={'type':'json_object'},
    messages=[{'role':'system','content':'Reply with JSON only. Keys: name (string), age (integer).'},
              {'role':'user','content':'My name is Ada and I am 36.'}])
import json; print(json.loads(resp.choices[0].message.content))

response_format json_object guarantees syntactic JSON, NOT shape; schema mode enforces shape.

Example 2

schema = {'name':'extract_invoice','schema':{'type':'object',
    'properties':{'items':{'type':'array','items':{
        'type':'object','properties':{
            'description':{'type':'string'},
            'amount':{'type':'number'}},
        'required':['description','amount']}}},
    'required':['items']},'strict':True}
resp = openai.chat.completions.create(
    model='gpt-4o-mini',
    response_format={'type':'json_schema','json_schema':schema},
    messages=[{'role':'user','content':'2 apples at $1.20 each, 1 milk at $3.50'}])

strict=True on json_schema makes the model commit to a single best-guess object ignoring refusal markers.

Example 3

import jsonschema, json
result = json.loads(resp.choices[0].message.content)
jsonschema.validate(result, schema['schema'])
print('validated', len(result['items']), 'items')

jsonschema.validate at the boundary — never trust LLM outputs blindly.

Common mistake

Trusting json_object mode to enforce shape — it only enforces syntactic JSON, not the keys you wanted. Without a schema the model can omit or invent keys.

Key takeaway

Use response_format json_object as the default for any JSON task. Upgrade to json_schema for strict shape. Validate at the boundary.

Production Failure Playbook

Failure scenario 1: json-mode-but-not-shape

Failure scenario 2: output-truncated-by-tokens