BaselineRubric-based evaluation: one definition of good
Rubric-based evaluation is how you make "good output" mean the same thing to everyone, so quality stops living in each reviewer's head. You write it down once as weighted criteria, and that Rubric becomes the single definition every Eval Run, Schedule, and Optimization Run measures against.
What it is, and why it matters
Ask three people whether an AI answer is "good" and you'll get three answers. One cares about accuracy, one about tone, one about length. That disagreement stays invisible until it ships as inconsistent quality, and it's why scores nobody defined are scores nobody trusts.
A rubric makes the standard explicit. You break "good" into named criteria and weight them by what actually matters to your product. Now everyone, and every automated grader, scores against the same thing. The fuzzy judgment becomes a shared, written artifact your team owns.
Because the rubric is one reusable object, it ties the whole workflow together. The same criteria that define a passing Eval Run drive the scheduled checks and the optimization that fixes regressions. Change the definition of good in one place and everything downstream follows.
See it in Baseline, step by step
Set the scene
A rubric starts with a scenario description and an expected outcome in plain language: what the AI is being asked to do, and what a good answer achieves. Optional grounding context gives the judge reference material to check against.
Weight what matters
Add criteria and weight them so the overall score reflects your priorities. Scoring steps under each criterion tell the judge exactly how to grade it, in your team's words.
One rubric, every measurement
The finished rubric drives one-off Eval Runs, recurring Schedules, and Optimization Runs alike. Edit the definition once and everything downstream measures against the update.
How Baseline does it
Scenario and expected outcome
The rubric captures the task and the target in prose first, so the criteria have context and a new teammate can read what good means here.
Weighted criteria
Name the dimensions of quality and weight them by importance. Weights sum to 1, so priorities are explicit and the overall score reflects them.
Scoring steps
Each criterion carries the step-by-step instructions the judge follows, turning a label like Accuracy into a repeatable procedure.
Team ownership
Contributors author and edit in the browser; Readonly Members see every result while the standard stays stable.
What you get
- Every reviewer, human or automated, scores against one written standard.
- Quality becomes an explicit written artifact the team owns.
- Domain experts define good directly, in the browser.
- One rubric powers evaluation, monitoring, and optimization.
Frequently asked questions
- What exactly is a rubric here?
- A set of weighted criteria that define a good output, authored in plain language in the browser. It's the single, shared standard every evaluation, schedule, and optimization scores against.
- Who writes the rubric?
- The person who knows what good looks like, usually a domain expert or product owner rather than an engineer. Baseline is built so they can author and edit it directly in the UI.
- Can I change the criteria later?
- Yes. Edit the Rubric, and every Eval Run, Schedule, and Optimization Run that references it measures against the updated definition. One change, applied everywhere.