PromptEvalScenario
A scenario of an evaluation suite (1.55.0): a simulated caller with a rubric, or a scripted conversation test with expectations, with synthetic variables and Tool mocks.
An object with 16 fields.
Fields
| Field | Type | Required | Description |
|---|---|---|---|
caller | PromptEvalCaller or null | yes | The caller, the content class text: null unless the read names include=text, and null in the answer of a write. |
created_at | Timestamp | yes | |
expectations | array of ConversationTestExpectation | yes | The expectations of a scripted scenario; empty otherwise. |
id | PromptEvalScenarioId | yes | |
initial_slots | PromptEvalInitialSlots | no | The starting slot values (1.63.0); absent from an api before 1.63.0, which a reader treats as none. |
kind | PromptEvalScenarioKind | yes | |
max_turns | integer | yes | The most Turns a Call of the scenario takes before it is ended. |
name | string | yes | A display name. |
object | "prompt_eval_scenario" | yes | |
revision | integer | yes | Counts writes; returned as the ETag and required in If-Match. |
rubric | array of PromptEvalRubricItem | yes | The rubric of a simulated scenario; empty otherwise. |
suite_id | PromptEvalSuiteId | yes | |
tags | array of PromptEvalTag | yes | |
tool_mocks | array of PromptEvalToolMock | yes | |
updated_at | Timestamp | yes | |
variables | PromptEvalVariables | yes |
Used by
The operations whose parameters, request body or responses name this shape directly.