LlmOptions

Language model request options.

An object with 4 fields.

Fields

prompt_cachingone of "auto", "off"

Provider prompt caching on the stable prefix, where the model offers it.

thinking_budget_tokensinteger

Extended thinking budget where the model offers it; 0 turns it off.

tool_choiceToolChoice

top_pnumber

Nucleus sampling.