Structured output and reasoning

Structured output

Use StructuredOutputFormat<T> to pair a JSON Schema with runtime parsing. The same contract works with completions, Responses, and Messages:

import { type StructuredOutputFormat } from "any-llm-ts";

interface Location {
  city: string;
}

const locationFormat: StructuredOutputFormat<Location> = {
  name: "location",
  jsonSchema: {
    type: "object",
    properties: { city: { type: "string" } },
    required: ["city"],
    additionalProperties: false,
  },
  parse(value) {
    if (typeof value !== "object" || value === null || !("city" in value)) {
      throw new TypeError("Invalid location output");
    }
    return { city: String(value.city) };
  },
};

const response = await llm.completion({
  model: "gpt-4.1-mini",
  messages: [{ role: "user", content: "Extract the city: I live in Kolkata." }],
  responseFormat: locationFormat,
});

console.log(response.choices[0]?.message.parsed?.city);

Use responseFormat for completions and Responses, and outputFormat for Messages. Typed parsing returns a complete JSON value, so it is non-streaming. Anthropic and Otari can still stream schema-constrained Messages events when outputFormat is set; those streams are unparsed events rather than ParsedMessageResponse. LengthFinishReasonError and ContentFilterFinishReasonError preserve the partial normalized completion when parsing cannot finish for those reasons.

Provider-compatible formats

You can also pass a provider-compatible response format through the common responseFormat field:

const response = await llm.completion({
  model: "gpt-4.1-mini",
  messages: [{ role: "user", content: "Extract the city from: I live in Kolkata." }],
  responseFormat: {
    type: "json_schema",
    json_schema: {
      name: "location",
      strict: true,
      schema: {
        type: "object",
        properties: { city: { type: "string" } },
        required: ["city"],
        additionalProperties: false,
      },
    },
  },
});

For Anthropic, responseFormat.type must be json_schema, and the schema must be available at responseFormat.json_schema.schema. The adapter translates it to Anthropic's structured-output configuration.

For Anthropic Claude models on Bedrock, the adapter emulates json_schema output by forcing a private tool call and returning that tool's JSON input as normal assistant content. This works for direct and cross-region model IDs containing anthropic.. It is non-streaming and cannot be combined with enabled extended reasoning; omit reasoningEffort or use none or auto.

Gemini accepts json_schema, json_object, and text. For json_schema, the adapter sends the schema through the native Google Gen AI SDK and requests the application/json response MIME type. If Gemini reports truncation or content filtering before structured output is complete, the adapter rejects with ContextLengthExceededError or ContentFilterError instead of returning partial JSON.

With this lower-level form, the library returns model output as text. Parse and validate it in your application with the schema library of your choice.

Reasoning effort

const response = await llm.completion({
  model: "reasoning-model",
  messages: [{ role: "user", content: "Solve this problem..." }],
  reasoningEffort: "high",
});

Supported normalized values are none, minimal, low, medium, high, xhigh, max, and auto. Providers may support only a subset. Reasoning text, when exposed by the provider, is normalized to message.reasoning or delta.reasoning.