List a run's verdicts

GET /v1/prompt-eval-suites/{prompt_eval_suite_id}/runs/{run_id}/verdicts

Lists the verdicts of a run, one per scenario, arm and repetition, by scenario (1.55.0): the outcome, the score, each rubric item's verdict and confidence and each expectation's result. rationale is null unless include=rationale adds it, which needs transcripts:read and makes the read an audited content read; a rationale scrubbed at the Compliance profile's retention_days reads null.

Request

Set VOCAL_API to the base URL of your API and VOCAL_API_KEY to an API key with the prompts:read scope (Authentication and keys). Replace each placeholder, such as agt_... or <name>, with your own value.

curl "$VOCAL_API/v1/prompt-eval-suites/pes_.../runs/per_.../verdicts" \
  -H "Authorization: Bearer $VOCAL_API_KEY"

const response = await fetch(`${process.env.VOCAL_API}/v1/prompt-eval-suites/pes_.../runs/per_.../verdicts`, {
  headers: {
    Authorization: `Bearer ${process.env.VOCAL_API_KEY}`,
  },
});
console.log(response.status, await response.text());

import os

import requests

response = requests.get(
    f"{os.environ['VOCAL_API']}/v1/prompt-eval-suites/pes_.../runs/per_.../verdicts",
    headers={
        "Authorization": f"Bearer {os.environ['VOCAL_API_KEY']}",
    },
)
print(response.status_code, response.text)

Parameters

Responses

Fields of the 200 answer

The shape is PromptEvalVerdictList.