Domain 3 · 20%% of exam

Syntax errors vs semantic errors

The highest-value idea in this domain: schemas guarantee shape, never truth.

5 min read · Lesson 3 of 12 in this domain

This is the most valuable idea in the domain and the one most likely to be tested as a trap. A schema guarantees the shape of the output: the keys exist, the types match, the JSON parses. It says nothing whatsoever about whether the values are true. A perfectly schema-valid invoice can carry a total that no line item supports, and the API will return it without complaint because it never claimed to check that. Once you separate syntax from semantics, the design follows: schemas handle syntax, deterministic code handles arithmetic and business rules, and genuine ambiguity gets preserved for a human rather than collapsed by the model.

Key points

A JSON schema prevents syntax errors. It does nothing about semantic ones — a perfectly schema-valid response can contain entirely wrong values.
Classic case: extracted invoice line items that do not sum to the stated total. The fix is to extract both a stated_total and a calculated_total, compare them, and flag mismatches for human review.
Never let the model silently reconcile a discrepancy by adjusting values — that fabricates data no source supports.
A validation-retry loop is the deterministic layer: validate programmatically, and on failure re-prompt with the specific validation error. An identical blind retry gives the model no new information.
When sources genuinely conflict, capture all candidate values with their source locations and let downstream business logic reconcile. Forcing a premature collapse destroys the evidence.
Hard business rules belong in deterministic backend code, not in schema booleans the model populates and not in prompt text vulnerable to injection.

Diagram

Two different checks, two different layers

Worked example

Extract both numbers, never reconcile silently. An invoice lists items summing to $4,850 but states a total of $4,580 — a transposition in the source document. If your schema has one total field, the model must pick, and whichever it picks looks authoritative downstream. Instead give it stated_total and calculated_total. Your code compares them, sees a mismatch, and flags the document. You have converted an invisible data-integrity failure into an explicit exception, which is the only honest outcome when the source itself disagrees.

Exam trap

A requires_approval boolean on the tool schema looks like a control. The model sets it, so it is not one.

Check your understanding

Line items do not sum to the stated total. Best design?

Correct answer: B — Extract stated_total and calculated_total, compare, flag mismatches
Adjusting fabricates data no source supports; examples shift probabilities but cannot make arithmetic deterministic.

A JSON schema guarantees which of these?

Correct answer: B — That the output shape and types are valid
Schemas prevent syntax errors, never semantic ones.

Practise this domain with 20%%-weighted questions in the study app.

Open in study app

← Previous

JSON Schema support and limits

Prefill removal and its replacements

Source: Claude Docs — Structured outputs · Independent study aid, not affiliated with or endorsed by Anthropic.