PromptEvalRunCreateRequest
Runs the scenarios of a suite against a candidate and a baseline (1.55.0). Every issue is a detail of 422 validation_failed (PromptEvalRunIssueCode); a version that is not published is 409 agent_version_unavailable.
An object with 5 fields.
Fields
| Field | Type | Required | Description |
|---|---|---|---|
baseline | PromptEvalBaselineInput or PromptEvalBaselineChoice | no | A version, active or none; active when omitted. A choice rather than null, so a client that dumps unset fields as null cannot drop the baseline by accident. |
candidate | PromptEvalCandidateInput | yes | |
model_families | array of ModelFamily | no | The model families to evaluate (1.56.0; prompt-manager.md section 5 rule 3), each a model_family of an llm descriptor the candidate's chain reaches (prompt_eval_model_family_unreachable otherwise); the family of the chain's primary when omitted. From 1.63.0 each scenario and repetition runs once per family named. |
repetitions | integer | no | The suite's repetitions when omitted. |
scenario_ids | array of PromptEvalScenarioId | no | The scenarios to run; every scenario of the suite when omitted. |
Used by
The operations whose parameters, request body or responses name this shape directly.