Skip to the content

Read a calibration

GET /v1/prompt-eval-suites/{prompt_eval_suite_id}/calibrations/{calibration_id}

Property	Value
Operation	`get_prompt_eval_calibration`
Group	Prompt evaluations and experiments
Scope	`prompts:read`
Route class	`read`
Idempotency	No `Idempotency-Key`
Audit	Not audited: A metadata read; a label or a calibration holds ids, keys, verdict values and rates, never Call content.
Released in	1.63.0
Phase 0 host	Served (the hosted api pins 1.65.0)

Answers one calibration (1.63.0): its sample with each entry labelled or not, the judge compared, its status and job, and once completed the agreement per rubric kind with intervals and whether it met the target.

Request

Set VOCAL_API to the base URL of your API and VOCAL_API_KEY to an API key with the prompts:read scope (Authentication and keys). Replace each placeholder, such as agt_... or <name>, with your own value.

curl "$VOCAL_API/v1/prompt-eval-suites/pes_.../calibrations/pec_..." \
  -H "Authorization: Bearer $VOCAL_API_KEY"

const response = await fetch(`${process.env.VOCAL_API}/v1/prompt-eval-suites/pes_.../calibrations/pec_...`, {
  headers: {
    Authorization: `Bearer ${process.env.VOCAL_API_KEY}`,
  },
});
console.log(response.status, await response.text());

import os

import requests

response = requests.get(
    f"{os.environ['VOCAL_API']}/v1/prompt-eval-suites/pes_.../calibrations/pec_...",
    headers={
        "Authorization": f"Bearer {os.environ['VOCAL_API_KEY']}",
    },
)
print(response.status_code, response.text)

Parameters

Name	In	Type	Required	Description
`prompt_eval_suite_id`	path	PromptEvalSuiteId	yes	The id of an evaluation suite: `pes_` and the row's UUIDv7 in 26 Crockford base32 characters. Opaque and safe to log.
`calibration_id`	path	PromptEvalCalibrationId	yes	The id of a calibration of an evaluation suite's judge against human labels (1.63.0): `pec_` and the row's UUIDv7 in 26 Crockford base32 characters. Opaque and safe to log.

Responses

Status	Content type	Shape	Error codes
`200`	`application/json`	PromptEvalCalibration
`400`	`application/json`	ErrorEnvelope	`invalid_request`
`401`	`application/json`	ErrorEnvelope	`unauthenticated`
`403`	`application/json`	ErrorEnvelope	`forbidden`
`404`	`application/json`	ErrorEnvelope	`not_found`
`429`	`application/json`	ErrorEnvelope	`rate_limited`
`500`	`application/json`	ErrorEnvelope	`internal_error`

Fields of the `200` answer

The shape is PromptEvalCalibration.

Field	Type	Required	Description
`agreement`	PromptEvalAgreement or null	yes	Null until `completed`.
`cost_lines`	array of PromptEvalCostLine	yes
`created_at`	Timestamp	yes
`created_by`	ActorRef	yes
`error_code`	string or null	yes
`finished_at`	Timestamp or null	yes
`from_calibration_id`	PromptEvalCalibrationId or null	yes	The calibration whose labelled sample this one reuses; null when it drew its own.
`id`	PromptEvalCalibrationId	yes
`job_id`	JobId or null	yes	The `prompt.calibration` job that compares; null while `labelling`.
`judge`	StageRef	yes
`labelled`	integer	yes	The entries that carry a label.
`meets_target`	boolean or null	yes	Whether the agreement met the target (`calibrationMeetsTarget`); null until `completed`.
`object`	`"prompt_eval_calibration"`	yes
`run_id`	PromptEvalRunId	yes	The run the sample was drawn from.
`sample`	array of PromptEvalCalibrationEntry	yes
`started_at`	Timestamp or null	yes
`status`	PromptEvalCalibrationStatus	yes
`suite_id`	PromptEvalSuiteId	yes
`suite_revision`	integer	yes
`target`	PromptEvalCalibrationTarget	yes