Skip to the content

List a suite's scenarios

GET /v1/prompt-eval-suites/{prompt_eval_suite_id}/scenarios

Property	Value
Operation	`list_prompt_eval_scenarios`
Group	Prompt evaluations and experiments
Scope	`prompts:read`
Scopes of `include` values	`text` needs `prompts:read`
Route class	`read`
Idempotency	No `Idempotency-Key`
Audit	An audited content read `prompts.prompt_eval_scenario.list`, when the request names an `include` value
Released in	1.55.0
Phase 0 host	Served (the hosted api pins 1.65.0)

Lists the scenarios of a suite, newest first, optionally by kind or tag (1.55.0); caller is null unless include=text adds it, which makes the list an audited content read.

Request

Set VOCAL_API to the base URL of your API and VOCAL_API_KEY to an API key with the prompts:read scope (Authentication and keys). Replace each placeholder, such as agt_... or <name>, with your own value.

curl "$VOCAL_API/v1/prompt-eval-suites/pes_.../scenarios" \
  -H "Authorization: Bearer $VOCAL_API_KEY"

const response = await fetch(`${process.env.VOCAL_API}/v1/prompt-eval-suites/pes_.../scenarios`, {
  headers: {
    Authorization: `Bearer ${process.env.VOCAL_API_KEY}`,
  },
});
console.log(response.status, await response.text());

import os

import requests

response = requests.get(
    f"{os.environ['VOCAL_API']}/v1/prompt-eval-suites/pes_.../scenarios",
    headers={
        "Authorization": f"Bearer {os.environ['VOCAL_API_KEY']}",
    },
)
print(response.status_code, response.text)

Parameters

Name	In	Type	Required	Description
`prompt_eval_suite_id`	path	PromptEvalSuiteId	yes	The id of an evaluation suite: `pes_` and the row's UUIDv7 in 26 Crockford base32 characters. Opaque and safe to log.
`limit`	query	integer	no	The page size, 1 to 100.
`cursor`	query	string	no	The opaque `next_cursor` of the previous page; it binds the filters it was issued under.
`kind`	query	PromptEvalScenarioKind	no	Only the scenarios of this kind.
`tag`	query	PromptEvalTag	no	Only the scenarios carrying this tag.
`include`	query	array of PromptEvalScenarioInclude	no	Content classes to add, comma-separated: `text` answers `caller` and makes the read an audited content read (1.55.0).

Responses

Status	Content type	Shape	Error codes
`200`	`application/json`	PromptEvalScenarioList
`400`	`application/json`	ErrorEnvelope	`invalid_request`, `invalid_cursor`
`401`	`application/json`	ErrorEnvelope	`unauthenticated`
`403`	`application/json`	ErrorEnvelope	`forbidden`
`404`	`application/json`	ErrorEnvelope	`not_found`
`429`	`application/json`	ErrorEnvelope	`rate_limited`
`500`	`application/json`	ErrorEnvelope	`internal_error`

Fields of the `200` answer

The shape is PromptEvalScenarioList.

Field	Type	Required	Description
`data`	array of PromptEvalScenario	yes	The scenarios of this page, newest first.
`has_more`	boolean	yes	True when a further page exists.
`next_cursor`	string or null	yes	The cursor of the next page, or null on the last page.
`object`	`"list"`	yes