Skip to content

Score response faithfulness to curb AI hallucinations

Run automated faithfulness evals checking every generated claim against retrieved context docs before responses ship to users.

Copy the prompt and paste it into the AI assistant you already use.

Steps to curb hallucinations with evals

  1. 01

    Include unanswerable trap questions

    Add 15 test cases where the provided context intentionally omits the answer to test honest refusal.

  2. 02

    Demand source quote grounding

    Require a supporting_excerpt field in schemas before outputting factual assertions.

  3. 03

    Audit numerical claims strictly

    Verify dates, pricing, and configuration flags match retrieved snippets character for character.

  4. 04

    Keep retrieval chunks clean

    Inspect whether low faithfulness stems from bad prompt rules or missing retrieval context.

Common questions

How do faithfulness evals detect hallucinations automatically?

Automated judge checks compare every claim in the endpoint's output solely against the input context supplied during that request.

Can faithfulness scoring run asynchronously on live traffic?

Yes. Run full faithfulness suites pre-deploy, and optionally sample a percentage of live production traces asynchronously.

What usually causes low faithfulness scores?

Either prompts lack explicit instructions to admit missing information, or retrieval connectors return noisy off-topic chunks.