Skip to the content

Update a scenario

PATCH /v1/prompt-eval-suites/{prompt_eval_suite_id}/scenarios/{scenario_id}

Property	Value
Operation	`update_prompt_eval_scenario`
Group	Prompt evaluations and experiments
Scope	`prompts:write`
Route class	`control`
Idempotency	No `Idempotency-Key`
Audit	Audited `prompts.prompt_eval_scenario.update` (mutation)
Released in	1.55.0
Phase 0 host	Served (the hosted api pins 1.65.0)

Replaces the named fields of a scenario at the revision If-Match names, checked as a creation is on the merged scenario (1.55.0); kind is fixed at create.

Request

Set VOCAL_API to the base URL of your API and VOCAL_API_KEY to an API key with the prompts:write scope (Authentication and keys). Replace each placeholder, such as agt_... or <name>, with your own value.

curl -X PATCH "$VOCAL_API/v1/prompt-eval-suites/pes_.../scenarios/pesc_..." \
  -H "Authorization: Bearer $VOCAL_API_KEY" \
  -H 'If-Match: "1"' \
  -H "Content-Type: application/json" \
  -d '{}'

const response = await fetch(`${process.env.VOCAL_API}/v1/prompt-eval-suites/pes_.../scenarios/pesc_...`, {
  method: 'PATCH',
  headers: {
    Authorization: `Bearer ${process.env.VOCAL_API_KEY}`,
    'If-Match': '"1"',
    'Content-Type': 'application/json',
  },
  body: JSON.stringify({}),
});
console.log(response.status, await response.text());

import os

import requests

response = requests.patch(
    f"{os.environ['VOCAL_API']}/v1/prompt-eval-suites/pes_.../scenarios/pesc_...",
    headers={
        "Authorization": f"Bearer {os.environ['VOCAL_API_KEY']}",
        "If-Match": '"1"',
    },
    json={},
)
print(response.status_code, response.text)

Parameters

Name	In	Type	Required	Description
`prompt_eval_suite_id`	path	PromptEvalSuiteId	yes	The id of an evaluation suite: `pes_` and the row's UUIDv7 in 26 Crockford base32 characters. Opaque and safe to log.
`scenario_id`	path	PromptEvalScenarioId	yes	The id of a scenario of an evaluation suite: `pesc_` and the row's UUIDv7 in 26 Crockford base32 characters. Opaque and safe to log.
`if-match`	header	string	yes	The revision the change was made against, quoted as the ETag returns it, or `*` to overwrite.

Request body

Content type application/json, required: PromptEvalScenarioUpdateRequest.

Field	Type	Required	Description
`caller`	PromptEvalCallerInput	no
`expectations`	array of ConversationTestExpectationInput	no
`initial_slots`	PromptEvalInitialSlots	no	The starting slot values (1.63.0); `{}` clears them.
`max_turns`	integer	no	The most Turns a Call of the scenario takes before it is ended.
`name`	string	no	A display name.
`rubric`	array of PromptEvalRubricItemInput	no
`tags`	array of PromptEvalTag	no
`tool_mocks`	array of PromptEvalToolMockInput	no
`variables`	PromptEvalVariables	no

Responses

Status	Content type	Shape	Error codes
`200`	`application/json`	PromptEvalScenario
`400`	`application/json`	ErrorEnvelope	`invalid_request`
`401`	`application/json`	ErrorEnvelope	`unauthenticated`
`403`	`application/json`	ErrorEnvelope	`forbidden`, `org_suspended`
`404`	`application/json`	ErrorEnvelope	`not_found`
`409`	`application/json`	ErrorEnvelope	`prompt_eval_suite_archived`
`412`	`application/json`	ErrorEnvelope	`revision_mismatch`
`422`	`application/json`	ErrorEnvelope	`validation_failed`
`428`	`application/json`	ErrorEnvelope	`precondition_required`
`429`	`application/json`	ErrorEnvelope	`rate_limited`
`500`	`application/json`	ErrorEnvelope	`internal_error`

Fields of the `200` answer

The shape is PromptEvalScenario.

Field	Type	Required	Description
`caller`	PromptEvalCaller or null	yes	The caller, the content class `text`: null unless the read names `include=text`, and null in the answer of a write.
`created_at`	Timestamp	yes
`expectations`	array of ConversationTestExpectation	yes	The expectations of a `scripted` scenario; empty otherwise.
`id`	PromptEvalScenarioId	yes
`initial_slots`	PromptEvalInitialSlots	no	The starting slot values (1.63.0); absent from an api before 1.63.0, which a reader treats as none.
`kind`	PromptEvalScenarioKind	yes
`max_turns`	integer	yes	The most Turns a Call of the scenario takes before it is ended.
`name`	string	yes	A display name.
`object`	`"prompt_eval_scenario"`	yes
`revision`	integer	yes	Counts writes; returned as the ETag and required in If-Match.
`rubric`	array of PromptEvalRubricItem	yes	The rubric of a `simulated` scenario; empty otherwise.
`suite_id`	PromptEvalSuiteId	yes
`tags`	array of PromptEvalTag	yes
`tool_mocks`	array of PromptEvalToolMock	yes
`updated_at`	Timestamp	yes
`variables`	PromptEvalVariables	yes