BaselineReduce AI hallucinations before they reach customers
A hallucination is an answer your AI gives confidently that simply isn't true. You can't stop a model from ever making one, but you can measure how often it happens, catch new ones before they ship, and steadily push the rate down. Baseline gives you the rubric, the scheduled checks, and the optimization loop to do exactly that.
What it is, and why it matters
Hallucinations are dangerous because they're confident. The model doesn't flag the answer as a guess, so a wrong price, a made-up policy, or an invented citation reads exactly like a correct one. By the time a customer notices, the damage is done.
You reduce hallucinations the way you fix any quality problem you can't see: you make it measurable. Define what a grounded, accurate answer looks like, score real outputs against that definition, and "how often does our AI make things up?" becomes a number you can watch rather than a feeling you argue about.
Once the rate is measured, you can act on it. Scheduled checks catch a new spike the day a prompt or model changes, and an optimization pass rewrites the prompts that produce the most slips. The number comes down, and you can prove it.
See it in Baseline, step by step
Write criteria that reward grounded answers
Give accuracy the heaviest weight and spell out the scoring steps: compare against the expected answer, penalize invented facts. Grounding context hands the judge the reference material to check claims against.
Score a real batch and see where it strays
Run an eval over real outputs. The per-row breakdown shows which answers slipped, and the judge's reasoning names the exact claim that cost the points.
Put the check on a schedule
A nightly or hourly Schedule re-scores fresh outputs from your live System, so a spike in made-up answers surfaces on the very next tick.
Drive the rate down and prove it
The dashboard shows the accuracy trend. When it dips, an Optimization Run searches for prompts that hold the line and reports the recovery as a number.
How Baseline does it
Accuracy-first Rubrics
Criteria that reward grounded, verifiable answers, weighted so accuracy dominates the overall score.
Grounding context
Attach the reference material the judge checks claims against, so "true" means true to your own docs and policies.
A measured rate
Each run turns a batch into a number, and the per-row reasoning names each invented fact it found.
Checks that keep running
Schedules re-score live outputs on a cadence; a model or prompt change that starts slipping shows up in the next run.
What you get
- Put a real number on how often your AI makes things up.
- See the exact claims that failed, with the judge's reasoning.
- Catch a spike within one scheduled run of it starting.
- Show the accuracy improvement as a trend, with the receipts.
Frequently asked questions
- Can you actually stop an LLM from hallucinating?
- Not entirely, and anyone promising zero is overselling. What you can do is measure the rate, catch regressions early, and drive it down with better prompts and grounding. Baseline is built for that loop.
- How do you measure something as fuzzy as a hallucination?
- You define what a grounded, accurate answer looks like as rubric criteria, then score outputs against it. The fuzzy worry becomes a number you can track over time.
- What makes a rubric good at catching hallucinations?
- Three things: an expected outcome the judge can compare against, grounding context that supplies the true reference material, and scoring steps that explicitly penalize invented facts. The walkthrough above sets up all three.