Configuration

Configuration can happen at three levels: provider discovery, SDK client construction, and an individual request.

Provider configuration

AnyLLM.create() accepts a provider name and ProviderOptions:

import { AnyLLM } from "any-llm-ts";

const llm = AnyLLM.create("openai", {
  apiKey: process.env.OPENAI_API_KEY,
  apiBase: "https://api.openai.com/v1",
  clientOptions: {
    maxRetries: 3,
    timeout: 30_000,
  },
});

The values in clientOptions are intentionally typed as Record<string, unknown> because the accepted keys differ between SDKs and versions.

Request-specific fields

Use providerOptions for request properties not represented by the common interface:

await llm.completion({
  model: "provider-model-name",
  messages: [{ role: "user", content: "Hello" }],
  providerOptions: {
    service_tier: "priority",
  },
});

providerOptions is merged into the outgoing payload after normalized fields, so it can override wire-level values. That is useful as an escape hatch, but it also means you should validate or hard-code these options rather than accepting them directly from untrusted callers.

Azure OpenAI

Azure OpenAI requires both an endpoint and an API key:

export AZURE_OPENAI_ENDPOINT="https://example.openai.azure.com"
export AZURE_OPENAI_API_KEY="..."
export OPENAI_API_VERSION="2024-10-21" # optional

const azure = AnyLLM.create("azureopenai");

The default API version is 2024-10-21. You can pass Azure SDK constructor fields through clientOptions when necessary.

Inspect effective metadata

const metadata = AnyLLM.getProviderMetadata("openai");

console.log(metadata.envApiKey);
console.log(metadata.apiBase);
console.log(metadata.capabilities.streaming);

Metadata is returned as a clone, so changing the returned object does not modify the registry.