BlogAI engineering

AI engineering

Structured AI Output Still Needs Business Validation

A valid JSON response is only the first check. Validate schemas, business rules, source evidence, and error handling before AI output changes application data.

Structured output makes a model response easier for software to consume, but syntactic correctness does not establish factual correctness. JSON Schema can express properties, required fields, and restrictions on additional fields. A production feature also needs rules that connect those fields to business meaning, permissions, and the evidence available for the task.

Define uncertainty in the schema

Design fields so the model can represent missing or ambiguous information honestly. If an invoice does not show a purchase order number, the schema should allow an explicit absent value rather than pressuring the model to invent one. Separate extracted text from normalized values when that distinction helps review.

Use constrained choices for known categories and clear types for amounts, identifiers, and dates. Decide whether unknown fields should be rejected or ignored. Keep the schema version with stored output, especially when downstream jobs may process results created before a deployment. A schema is an interface shared by multiple parts of the system.

Apply domain rules after parsing

Imagine a hypothetical purchasing application extracting a supplier quote. The response may be valid JSON while its line totals do not match the quoted subtotal. Application code should check arithmetic, currency consistency, expected identifiers, and required evidence. These deterministic checks are often easier to explain than asking another model whether the object looks right.

Classify failures carefully. A missing required field is different from a total mismatch or an unsupported supplier identifier. Give the user a correction path that preserves the original source and highlights the questionable fields. Keep any automatic repair attempts bounded and log that a repair occurred so reviewers can reconstruct the result.

Control what happens after validation

Choose separate states for parsed, validated, reviewed, and applied when the workflow needs them. Do not treat successful parsing as permission to update a record. Recheck the user’s authority and the current record version before applying an action derived from model output.

Test empty input, conflicting dates, unexpected units, extra fields, very long strings, and a response that stops midway. Also test a plausible but incorrect object that satisfies the schema. That last case demonstrates why evidence checks and review design matter. Record validation failures by category so recurring issues can inform prompt changes or improvements to source extraction.

Sources & further reading