BlogTesting & quality

Testing & quality

Evaluate unsupported claims in an AI feature

Check whether AI answers are supported by available evidence. A practical evaluation separates invented facts, missing context, and appropriately uncertain responses.

A fluent answer can contain an unsupported claim even when most of its content is useful. Evaluating hallucinations starts by defining what evidence the feature is allowed to use. Consider a hypothetical internal policy assistant. It may summarize approved documents, but it should not fill gaps with an invented reimbursement amount. The test should examine the connection between each consequential claim and its supporting material.

Define what counts as support

Choose the relevant evidence boundary for the product. An internal policy answer might require support from current policy documents, while a creative brainstorming tool has different expectations. Break a sample response into individual factual claims. Ask whether the available source supports each claim, contradicts it, or says nothing about it. A citation alone is insufficient: the linked passage must actually substantiate the statement being made.

Design examples with deliberate gaps

Include complete evidence, partial evidence, conflicting revisions, and no relevant evidence. In the reimbursement example, supply a policy that describes eligible travel without mentioning a daily limit. A successful answer should identify that gap and direct the user toward the next approved step. This reveals whether the system understands its evidence boundary more clearly than a dataset containing only questions with obvious answers.

Score consequence as well as frequency

Record the type of unsupported claim and the decision it could affect. Inventing a policy amount is different from adding an unnecessary adjective. Use a severity label that the product owner understands and report serious failures separately. Avoid presenting one percentage as a complete assessment. Inspect whether failures concentrate around particular document types, retrieval gaps, or ambiguous user questions, because those patterns suggest different engineering fixes.

Improve the right part of the system

A retrieval problem may need better document selection, while unsupported extrapolation may need a clearer answer contract. Conflicting sources may require revision metadata and a visible warning. Recheck the whole task after making a change. Excessive refusal can eliminate unsupported claims while making the assistant unhelpful. Track whether the user receives a supported answer, a useful clarification, or an honest explanation of what the system cannot establish.

Sources & further reading