Create an experiment on an Agent
POST /v1/agents/{agent_id}/experiments
| Property | Value |
|---|---|
| Operation | create_experiment |
| Group | Prompt evaluations and experiments |
| Scope | agents:write |
| Route class | control |
| Idempotency | Idempotency-Key accepted: a retry with the same key replays the first answer |
| Audit | Audited agent.experiment.create (mutation) |
| Released in | 1.55.0 |
| Phase 0 host | Served (the hosted api pins 1.65.0) |
Creates a traffic-split experiment as a draft with one or two candidate versions and their shares, the control receiving the rest once started (1.55.0; prompt-manager.md section 9.2). A candidate whose Policy versions, Tools or Knowledge bindings differ from the Agent's active version is experiment_policy_mismatch, and one that is the active version experiment_arm_is_control, each a detail of 422 validation_failed; a candidate version that is not published is 409 agent_version_unavailable; a suite_id is refused with 422 capability_not_available and live_call_judging_not_available until the Compliance profile carries live_call_judging.
Request
Set VOCAL_API to the base URL of your API and VOCAL_API_KEY to an API key with the agents:write scope (Authentication and keys). Replace each placeholder, such as agt_... or <name>, with your own value.
curl -X POST "$VOCAL_API/v1/agents/agt_.../experiments" \
-H "Authorization: Bearer $VOCAL_API_KEY" \
-H "Idempotency-Key: $(uuidgen)" \
-H "Content-Type: application/json" \
-d '{
"name": "<name>",
"candidates": [
{
"agent_version": 1,
"share_percent": 1
}
]
}'const response = await fetch(`${process.env.VOCAL_API}/v1/agents/agt_.../experiments`, {
method: 'POST',
headers: {
Authorization: `Bearer ${process.env.VOCAL_API_KEY}`,
'Idempotency-Key': crypto.randomUUID(),
'Content-Type': 'application/json',
},
body: JSON.stringify({
"name": "<name>",
"candidates": [
{
"agent_version": 1,
"share_percent": 1
}
]
}),
});
console.log(response.status, await response.text());import os
import uuid
import requests
response = requests.post(
f"{os.environ['VOCAL_API']}/v1/agents/agt_.../experiments",
headers={
"Authorization": f"Bearer {os.environ['VOCAL_API_KEY']}",
"Idempotency-Key": str(uuid.uuid4()),
},
json={
"name": "<name>",
"candidates": [
{
"agent_version": 1,
"share_percent": 1,
},
],
},
)
print(response.status_code, response.text)Parameters
| Name | In | Type | Required | Description |
|---|---|---|---|---|
agent_id | path | AgentId | yes | The id of an Agent: agt_ and the row's UUIDv7 in 26 Crockford base32 characters. Opaque and safe to log. |
idempotency-key | header | string | no | A client-chosen key, kept 24 hours per Organization, environment and route. A replay returns the stored response. |
Request body
Content type application/json, required: ExperimentCreateRequest.
| Field | Type | Required | Description |
|---|---|---|---|
candidates | array of ExperimentCandidateInput | yes | |
description | string | no | For reviewers. |
goal | ExperimentGoalInput | no | The goal the results report per arm (1.63.0): a slot value at Call end or an Outcome field value. Fixed at create. None when omitted. |
name | string | yes | A display name. |
stop_rules | ExperimentStopRulesInput | no | |
suite_id | PromptEvalSuiteId | no | A suite whose rubric judges a sample of live Calls, only with live_call_judging; refused with capability_not_available and live_call_judging_not_available until the Compliance profile carries it, and from 1.63.0 while the profile has it off. |
unit | ExperimentUnit | no | call when omitted. |
Responses
| Status | Content type | Shape | Error codes |
|---|---|---|---|
201 | application/json | Experiment | |
400 | application/json | ErrorEnvelope | invalid_request |
401 | application/json | ErrorEnvelope | unauthenticated |
403 | application/json | ErrorEnvelope | forbidden, org_suspended |
404 | application/json | ErrorEnvelope | not_found |
409 | application/json | ErrorEnvelope | idempotency_in_progress, idempotency_key_reused, agent_version_unavailable |
422 | application/json | ErrorEnvelope | validation_failed, capability_not_available |
429 | application/json | ErrorEnvelope | rate_limited |
500 | application/json | ErrorEnvelope | internal_error |
Fields of the 201 answer
The shape is Experiment.
| Field | Type | Required | Description |
|---|---|---|---|
agent_id | AgentId | yes | |
arms | array of ExperimentArm | yes | The control first, then the candidates in the order they were named. |
created_at | Timestamp | yes | |
created_by | ActorRef | yes | |
description | string or null | yes | For reviewers. |
goal | ExperimentGoal or null | no | The goal the results report per arm (1.63.0): a slot value at Call end or an Outcome field value. Fixed at create. Null for none; absent from an api before 1.63.0, which a reader treats as null. |
id | ExperimentId | yes | |
name | string | yes | A display name. |
object | "experiment" | yes | |
promoted_arm | ExperimentCandidateArmKey or null | yes | |
started_at | Timestamp or null | yes | |
started_by | ActorRef or null | yes | |
status | ExperimentStatus | yes | |
stop_reason | ExperimentStopReason or null | yes | |
stop_rule | ExperimentStopRule or null | yes | Set with stop_rule only. |
stop_rules | ExperimentStopRules | yes | |
stopped_at | Timestamp or null | yes | |
stopped_by | ActorRef or null | yes | Who stopped or promoted it; null when a stop rule or a version change did. |
suite_id | PromptEvalSuiteId or null | yes | A suite whose rubric judges a sample of live Calls, only with live_call_judging; refused with capability_not_available and live_call_judging_not_available until the Compliance profile carries it, and from 1.63.0 while the profile has it off. |
unit | ExperimentUnit | yes | |
updated_at | Timestamp | yes |