Run conversation tests against an Agent version

POST /v1/agents/{agent_id}/versions/{version}/tests

Runs the scripted scenarios of a suite of the Agent, or inline tests, against the Agent version as test Calls with purpose: "evaluation" on the headless Engine, each repeated as asked, as an agent.conversation_test job, and answers 202 with the run queued, its job_id and its Location (1.55.0). Nothing is judged by a model: an expectation is met or not by the recorded facts. A retired version is 409 agent_version_unavailable. Until a dedicated evaluation quota exists the Calls count against the daily allowance of test Calls (429 test_daily_cap_reached); from 1.63.0 they count against both, the evaluation quota refusing with 429 evaluation_quota_reached. Until the headless Engine runs evaluation Calls the request is refused with 422 capability_not_available and the detail evaluation_engine_not_available. The report is never a publish gate in 1.55.0; from 1.63.0 a run of a suite that defines a gate carries its outcome (gate), which an activation of the version reads once the owner of the Agent names the suite.

Request

Set VOCAL_API to the base URL of your API and VOCAL_API_KEY to an API key with the prompts:write scope (Authentication and keys). Replace each placeholder, such as agt_... or <name>, with your own value.

curl -X POST "$VOCAL_API/v1/agents/agt_.../versions/1/tests" \
  -H "Authorization: Bearer $VOCAL_API_KEY" \
  -H "Idempotency-Key: $(uuidgen)" \
  -H "Content-Type: application/json" \
  -d '{}'

const response = await fetch(`${process.env.VOCAL_API}/v1/agents/agt_.../versions/1/tests`, {
  method: 'POST',
  headers: {
    Authorization: `Bearer ${process.env.VOCAL_API_KEY}`,
    'Idempotency-Key': crypto.randomUUID(),
    'Content-Type': 'application/json',
  },
  body: JSON.stringify({}),
});
console.log(response.status, await response.text());

import os
import uuid

import requests

response = requests.post(
    f"{os.environ['VOCAL_API']}/v1/agents/agt_.../versions/1/tests",
    headers={
        "Authorization": f"Bearer {os.environ['VOCAL_API_KEY']}",
        "Idempotency-Key": str(uuid.uuid4()),
    },
    json={},
)
print(response.status_code, response.text)

Parameters

Request body

Content type application/json, required: ConversationTestRunRequest.

Responses

Fields of the 202 answer

The shape is ConversationTestRun.