PromptEvalVerdict
The verdicts of one evaluation Call (1.55.0): per rubric item for a simulated scenario, per expectation for a scripted one, with the outcome and the score. Content only in rationale, under include.
An object with 17 fields.
Fields
| Field | Type | Required | Description |
|---|---|---|---|
arm | PromptEvalArm | yes | |
call_id | CallId or null | yes | The evaluation Call, a test Call; null when none could start. |
ended_reason | EndedReason or null | yes | |
error_code | string or null | yes | |
expectations | array of ConversationTestExpectationResult | yes | |
judge | StageRef or null | yes | The judge; null for a scripted scenario. |
kind | PromptEvalScenarioKind | yes | |
model_family | ModelFamily or null | no | The model family the evaluation Call ran under (1.63.0); null for the family of the chain's primary on a run that named none. Absent from an api before 1.63.0. |
object | "prompt_eval_verdict" | yes | |
outcome | PromptEvalOutcome | yes | |
repetition | integer | yes | |
rubric | array of PromptEvalRubricVerdict | yes | |
run_id | PromptEvalRunId | yes | |
scenario_id | PromptEvalScenarioId | yes | |
scenario_revision | integer | yes | |
score | number or null | yes | The share of the applicable items and expectations that passed; null on error. |
turn_count | integer or null | yes |