PromptEvalRun

A run of an evaluation suite (1.55.0): the candidate and baseline, the per-arm pass rates with intervals and the deterministic facts, the judge and the cost lines. Ids, counts and rates only; the rationales are verdict content.

An object with 24 fields.

Fields

Field	Type	Required	Description
`advisory`	boolean	yes	True when the judged results gate nothing, because the judge is not checked against human labels (section 9.1 rule 5); true for every run of 1.55.0. From 1.63.0 false only when the suite's latest calibration met its target with the run's judge (`promptEvalJudgeAdvisory`), and false for a run that judged nothing.
`agent_id`	AgentId	yes
`baseline`	PromptEvalRunTarget or null	yes
`by_family`	array or null	no	One entry per evaluated family and arm (1.63.0), once `completed`; null before. Absent from an api before 1.63.0.
`by_item`	array or null	no	One entry per scenario, rubric item and arm (1.63.0), once `completed`; null before. Absent from an api before 1.63.0.
`candidate`	PromptEvalRunTarget	yes
`cost_lines`	array of PromptEvalCostLine	yes
`created_at`	Timestamp	yes
`created_by`	ActorRef	yes
`error_code`	string or null	yes
`finished_at`	Timestamp or null	yes
`gate`	PromptEvalGateOutcome or null	no	How the run fares against its suite's publish gate (1.63.0), once `completed`; null for a suite with no gate and before completion. Absent from an api before 1.63.0.
`id`	PromptEvalRunId	yes
`job_id`	JobId	yes	The `prompt.evaluation` job that runs it.
`judge`	StageRef or null	yes	The judge the run used; null with none.
`model_families`	array of ModelFamily	no	The model families the run evaluated (1.63.0): the request's `model_families`, or the family of the chain's primary. Absent from an api before 1.63.0.
`object`	`"prompt_eval_run"`	yes
`repetitions`	integer	yes
`scenario_count`	integer	yes
`started_at`	Timestamp or null	yes
`status`	PromptEvalRunStatus	yes
`suite_id`	PromptEvalSuiteId	yes
`suite_revision`	integer	yes
`summary`	array or null	yes	One entry per arm, the candidate first, once `completed`; null before.

Used by

The operations whose parameters, request body or responses name this shape directly.