> ## Documentation Index > Fetch the complete documentation index at: https://bastani-feat-pi-model-runtime-parity.mintlify.site/llms.txt > Use this file to discover all available pages before exploring further. # Models # Custom Models Add custom providers and models (Ollama, vLLM, LM Studio, proxies) via the single `models.json` in the active Atomic agent directory, normally `~/.atomic/agent/models.json`, or the directory selected by `ATOMIC_CODING_AGENT_DIR`/`PI_CODING_AGENT_DIR`. Atomic reads only that file: it does not read project-scoped `.atomic/models.json`, fall back to `~/.pi/agent/models.json`, or merge `.pi` and `.atomic` model configuration files. The legacy `.pi` read fallback remains available for configuration surfaces that explicitly use layered config paths, such as `auth.json`; it does not apply to `models.json`. A complete `defaultProvider`/`defaultModel` pair in `settings.json` is resolved after built-in, configured, and extension providers register. If the provider remains unsupported, interactive mode reports a generic saved-configuration warning and leaves model selection open instead of routing the session to a different provider. Print and JSON modes write that diagnostic to stderr and exit nonzero before prompting, keeping JSON stdout JSONL-clean. RPC rejects `prompt` with the same correlated diagnostic until an explicit successful `set_model` selects an available model or an explicit model cycle returns a different available model. A null or unchanged cycle result does not clear the condition. If the provider is supported but the model is unknown or lacks authentication, normal automatic selection of an available authenticated model continues. Valid custom- and extension-provider defaults resolve once their provider registration is available. See [Settings](/settings#model--thinking). ## Table of Contents * [Minimal Example](#minimal-example) * [Full Example](#full-example) * [Supported APIs](#supported-apis) * [Provider Configuration](#provider-configuration) * [Model Configuration](#model-configuration) * [Request-wide Cost Tiers](#request-wide-cost-tiers) * [Overriding Built-in Providers](#overriding-built-in-providers) * [Per-model Overrides](#per-model-overrides) * [Anthropic Messages Compatibility](#anthropic-messages-compatibility) * [OpenAI Compatibility](#openai-compatibility) ## Minimal Example For local models (Ollama, LM Studio, vLLM), only `id` is required per model: ```json theme={null} { "providers": { "ollama": { "baseUrl": "http://localhost:11434/v1", "api": "openai-completions", "apiKey": "ollama", "models": [ { "id": "llama3.1:8b" }, { "id": "qwen2.5-coder:7b" } ] } } } ``` The `apiKey` is required but Ollama ignores it, so any value works. Some OpenAI-compatible servers do not understand the `developer` role used for reasoning-capable models. For those providers, set `compat.supportsDeveloperRole` to `false` so Atomic sends the system prompt as a `system` message instead. If the server also does not support `reasoning_effort`, set `compat.supportsReasoningEffort` to `false` too. You can set `compat` at the provider level to apply to all models, or at the model level to override a specific model. This commonly applies to Ollama, vLLM, SGLang, and similar OpenAI-compatible servers. ```json theme={null} { "providers": { "ollama": { "baseUrl": "http://localhost:11434/v1", "api": "openai-completions", "apiKey": "ollama", "compat": { "supportsDeveloperRole": false, "supportsReasoningEffort": false }, "models": [ { "id": "gpt-oss:20b", "reasoning": true } ] } } } ``` ## Full Example Override defaults when you need specific values: ```json theme={null} { "providers": { "ollama": { "baseUrl": "http://localhost:11434/v1", "api": "openai-completions", "apiKey": "ollama", "models": [ { "id": "llama3.1:8b", "name": "Llama 3.1 8B (Local)", "reasoning": false, "input": ["text"], "contextWindow": 128000, "maxTokens": 32000, "cost": { "input": 0, "output": 0, "cacheRead": 0, "cacheWrite": 0 } } ] } } } ``` Atomic reloads the active agent directory's single `models.json` each time you open `/model`. Provider definitions, per-model overrides, dynamic catalogs, and isolated-engine model state are rebuilt from that fresh configuration, so edits take effect without restarting. Invalid edits report an error. ## Google AI Studio Example Use `google-generative-ai` with a `baseUrl` to add models from Google AI Studio, including custom Gemma 4 entries: ```json theme={null} { "providers": { "my-google": { "baseUrl": "https://generativelanguage.googleapis.com/v1beta", "api": "google-generative-ai", "apiKey": "$GEMINI_API_KEY", "models": [ { "id": "gemma-4-31b-it", "name": "Gemma 4 31B", "input": ["text", "image"], "contextWindow": 262144, "reasoning": true } ] } } } ``` The `baseUrl` is required when adding custom models to the `google-generative-ai` API type. ## Supported APIs | API | Description | | ---------------------- | ----------------------------------------- | | `openai-completions` | OpenAI Chat Completions (most compatible) | | `openai-responses` | OpenAI Responses API | | `anthropic-messages` | Anthropic Messages API | | `google-generative-ai` | Google Generative AI | Set `api` at provider level (default for all models) or model level (override per model). These four values are the generic custom-provider APIs supported by `models.json`. Atomic's installed native provider runtime also implements provider-owned APIs including `mistral-conversations`, `azure-openai-responses`, `openai-codex-responses`, `bedrock-converse-stream`, `google-vertex`, and `pi-messages`; those native APIs are not implied to be stable generic custom-provider contracts. ## Provider Configuration | Field | Description | | ---------------- | -------------------------------------------------------------------------------------------------------------- | | `baseUrl` | API endpoint or gateway URL | | `api` | Generic custom-provider API type (see above) | | `apiKey` | Optional API key (see value resolution below); omit when auth comes from `/login`, `auth.json`, or `--api-key` | | `oauth` | Dynamic OAuth provider type. Currently `"radius"`; requires the gateway `baseUrl` | | `headers` | Custom headers (see value resolution below) | | `authHeader` | Set `true` to add `Authorization: Bearer ` automatically | | `models` | Array of model configurations | | `modelOverrides` | Per-model overrides for matching built-in or extension-registered models on this provider | For a custom Radius gateway, set `"oauth": "radius"` and its `baseUrl`. Atomic uses Radius OAuth credentials and the gateway's dynamic `pi-messages` catalog. ### Value Resolution The `apiKey` and `headers` fields support three formats: * **Shell command:** `"!command"` executes and uses stdout ```json theme={null} "apiKey": "!security find-generic-password -ws 'anthropic'" "apiKey": "!op read 'op://vault/item/credential'" ``` * **Environment variable:** Prefix the variable name with `$` (or use `${VAR}`) to resolve it from the environment ```json theme={null} "apiKey": "$MY_API_KEY" ``` * **Literal value:** Used directly when the value does not use shell-command or explicit environment-variable syntax. Use `$MY_API_KEY`/`${MY_API_KEY}` for new environment-variable references; legacy uppercase env-var-like values may be migrated as described below. ```json theme={null} "apiKey": "sk-..." ``` Legacy uppercase env-var-like values in existing `models.json` provider config, such as `MY_API_KEY`, are migrated to `$MY_API_KEY` on startup only when that environment variable is present during migration; otherwise the value is preserved as a literal. New configs should use explicit `$ENV_VAR`/`${ENV_VAR}` syntax for environment variables. For `models.json`, shell commands are resolved at request time. Atomic intentionally does not apply built-in TTL, stale reuse, or recovery logic for arbitrary commands. Different commands need different caching and failure strategies, and Atomic cannot infer the right one. If your command is slow, expensive, rate-limited, or should keep using a previous value on transient failures, wrap it in your own script or command that implements the caching or TTL behavior you want. `/model` availability checks use configured auth presence and do not execute shell commands. ### Custom Headers ```json theme={null} { "providers": { "custom-proxy": { "baseUrl": "https://proxy.example.com/v1", "apiKey": "$MY_API_KEY", "api": "anthropic-messages", "headers": { "x-portkey-api-key": "$PORTKEY_API_KEY", "x-secret": "!op read 'op://vault/item/secret'" }, "models": [...] } } } ``` ## Model Configuration | Field | Required | Default | Description | | ------------------- | -------- | ----------------- | ------------------------------------------------------------------------------------------------------------ | | `id` | Yes | — | Model identifier (passed to the API) | | `name` | No | `id` | Human-readable model label. Used for matching (`--model` patterns) and shown as secondary model detail text. | | `api` | No | provider's `api` | Override provider's API for this model | | `reasoning` | No | `false` | Supports extended thinking | | `thinkingLevelMap` | No | omitted | Maps Atomic thinking levels to provider values and marks unsupported levels (see below) | | `input` | No | `["text"]` | Input types: `["text"]` or `["text", "image"]` | | `contextWindow` | No | `128000` | Default/effective context window size in tokens | | `maxTokens` | No | `16384` | Maximum output tokens | | `cost` | No | all zeros | Complete base rates per million tokens plus optional request-wide `tiers` (see below) | | `compat` | No | provider `compat` | Provider compatibility overrides. Merged with provider-level `compat` when both are set. | | `deferredToolsMode` | No | omitted | Deferred tool-loading protocol; set to `"kimi"` for Kimi-compatible deferred tools | Current behavior: * `/model`, `--list-models`, and the interactive footer display entries by model `id`. * The configured `name` is used for model matching and secondary model detail text. It does not replace the footer/status-bar model id. Model references resolve the complete, unmodified ID before Atomic interprets thinking suffixes or glob syntax. For example, if the catalog contains the literal ID `provider/literal[free]:high`, that complete model wins and `:high` remains part of its ID; it does not become a thinking-level suffix and `[free]` is not treated as a character class. Only when the complete ID is absent does Atomic parse a valid thinking suffix, try the stripped exact ID, then apply glob/fuzzy matching. This preserves literal provider IDs without changing ordinary `*`, `?`, bracket-glob, ambiguity, ordering, or deduplication behavior. ### Request-wide Cost Tiers Custom models can declare request-wide long-context pricing under `cost.tiers`. The base `cost` and every tier must provide all four rates: `input`, `output`, `cacheRead`, and `cacheWrite`, in cost per million tokens. Each tier also requires `inputTokensAbove`. ```json theme={null} { "id": "long-context-model", "cost": { "input": 1, "output": 2, "cacheRead": 0.25, "cacheWrite": 0.5, "tiers": [ { "inputTokensAbove": 272000, "input": 2, "output": 3, "cacheRead": 0.5, "cacheWrite": 1 } ] } } ``` Atomic chooses one rate set for the entire request. It calculates aggregate input as `input + cacheRead + cacheWrite`, selects only tiers whose threshold is **strictly exceeded**, and uses the matching tier with the highest `inputTokensAbove`. Exactly 272,000 aggregate input tokens in the example still use the base rates; 272,001 use every rate from the tier, including the tier's output rate. For `modelOverrides`, `cost` is partial: any supplied scalar rate replaces that scalar while omitted scalar rates remain inherited. A scalar-only cost override also preserves inherited tiers. Supplying `tiers` replaces the whole inherited tier array; use `"tiers": []` to clear it explicitly. Every supplied replacement tier must still be complete. ```json theme={null} { "providers": { "openai": { "modelOverrides": { "gpt-5.6-sol": { "cost": { "input": 4, "tiers": [] } } } } } } ``` This override changes only the base input rate, retains the model's other base rates, and clears its inherited long-context tiers. ### Thinking Level Map Use `thinkingLevelMap` on a model to describe model-specific thinking controls. Keys are Atomic thinking levels: `off`, `minimal`, `low`, `medium`, `high`, `xhigh`, `max`. A level is selectable only when the active model supports it; `xhigh` and `max` are not universal provider capabilities. Values are tristate: | Value | Meaning | | ------- | ---------------------------------------------------------- | | omitted | Level is supported and uses the provider's default mapping | | string | Level is supported and this value is sent to the provider | | `null` | Level is unsupported and hidden/skipped/clamped away | Example for a model that only supports off, high, and max reasoning: ```json theme={null} { "id": "deepseek-v4-pro", "reasoning": true, "thinkingLevelMap": { "minimal": null, "low": null, "medium": null, "high": "high", "xhigh": null, "max": "max" } } ``` Example for a model where thinking cannot be disabled: ```json theme={null} { "id": "always-thinking-model", "reasoning": true, "thinkingLevelMap": { "off": null } } ``` Migration: older configs that used `compat.reasoningEffortMap` should move that mapping to model-level `thinkingLevelMap`. Use `null` for levels that should not appear in the UI. ### Context Window `contextWindow` is the model's context size in tokens and drives local budgeting, compaction thresholds, footer/stats, session replay, and RPC/SDK state. ```json theme={null} { "id": "long-context-model", "reasoning": true, "contextWindow": 400000 } ``` Built-in models take their `contextWindow` from the bundled `pi-ai` catalog. To change one, use `modelOverrides`: ```json theme={null} { "providers": { "github-copilot": { "modelOverrides": { "gpt-5.5": { "contextWindow": 272000 } } } } } ``` To add a new model id under a built-in provider, define it in `models`: ```json theme={null} { "providers": { "github-copilot": { "models": [ { "id": "my-copilot-model", "contextWindow": 400000 } ] } } } ``` ## Overriding Built-in Providers Route a built-in provider through a proxy without redefining models: ```json theme={null} { "providers": { "anthropic": { "baseUrl": "https://my-proxy.example.com/v1" } } } ``` All built-in Anthropic models remain available. Existing OAuth or API key auth continues to work. To merge custom models into a built-in provider, include the `models` array: ```json theme={null} { "providers": { "anthropic": { "baseUrl": "https://my-proxy.example.com/v1", "apiKey": "$ANTHROPIC_API_KEY", "api": "anthropic-messages", "models": [...] } } } ``` Merge semantics: * Built-in models are kept. * Custom models are upserted by `id` within the provider. * If a custom model `id` matches a built-in model `id`, the custom model replaces that built-in model. * If a custom model `id` is new, it is added alongside built-in models. ## Per-model Overrides Use `modelOverrides` to customize specific models without replacing the provider's full model list. Overrides apply to matching built-in models and to models later registered by an extension through `pi.registerProvider()`. ```json theme={null} { "providers": { "openrouter": { "modelOverrides": { "anthropic/claude-sonnet-4": { "name": "Claude Sonnet 4 (Bedrock Route)", "compat": { "openRouterRouting": { "only": ["amazon-bedrock"] } } } } } } } ``` `modelOverrides` supports these fields per model: `name`, `reasoning`, `thinkingLevelMap`, `input`, `cost` (partial scalar rates plus optional full tier-array replacement), `contextWindow`, `maxTokens`, `headers`, `compat`. Atomic reads one `models.json` from the active agent directory. It does not layer model overrides from `.pi` and `.atomic` files. Within a single file, custom model definitions replace matching built-in entries after built-in overrides are applied. `modelOverrides` composes only with built-in and extension-registered models; it does not modify a same-ID custom model definition. Behavior notes: * Atomic retains the parsed override map even when an extension registers the matching provider/model after `models.json` is loaded. * Model overrides come from the active agent directory's single `models.json`; no cross-file layering or merging is performed. * For matching built-in and extension-registered models, the model definition is the base and `modelOverrides` wins configured fields. Extension-registered model headers are shallow-merged with override headers, with override headers winning duplicate names. A same-ID custom model replaces the built-in override result, including its complete header record. * A scalar-only `cost` override preserves inherited tiers. Supplying `cost.tiers` replaces the complete tier array, including `[]` to clear it; omitted scalar cost fields remain inherited. * Provider-level request headers remain a separate provider layer and are combined at request time. * Unknown model IDs are ignored unless a matching model is subsequently registered by an extension. * If `models` is also defined for a provider in `models.json`, those custom models are merged after built-in overrides. A custom model with the same `id` replaces the overridden built-in model entry. ## Anthropic Messages Compatibility For providers or proxies using `api: "anthropic-messages"`, use `compat.supportsEagerToolInputStreaming` to control Anthropic fine-grained tool streaming compatibility. By default, Atomic sends per-tool `eager_input_streaming: true`. If a proxy or Anthropic-compatible backend rejects that field, set `supportsEagerToolInputStreaming` to `false`. Atomic will omit `tools[].eager_input_streaming` and send the legacy `fine-grained-tool-streaming-2025-05-14` beta header for tool-enabled requests instead. ```json theme={null} { "providers": { "anthropic-proxy": { "baseUrl": "https://proxy.example.com", "api": "anthropic-messages", "apiKey": "$ANTHROPIC_PROXY_KEY", "compat": { "supportsEagerToolInputStreaming": false, "supportsLongCacheRetention": true }, "models": [ { "id": "claude-opus-4-8", "reasoning": true, "input": ["text", "image"] } ] } } } ``` | Field | Description | | --------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | `supportsEagerToolInputStreaming` | Whether the provider accepts per-tool `eager_input_streaming`. Default: `true`. Set to `false` to omit that field and use the legacy fine-grained tool streaming beta header on tool-enabled requests. | | `supportsLongCacheRetention` | Whether the provider accepts Anthropic long cache retention (`cache_control.ttl: "1h"`) when cache retention is `long`. Default: `true`. | ## OpenAI Compatibility For providers with partial OpenAI compatibility, use the `compat` field. * Provider-level `compat` applies defaults to all models under that provider. * Model-level `compat` overrides provider-level values for that model. ```json theme={null} { "providers": { "local-llm": { "baseUrl": "http://localhost:8080/v1", "api": "openai-completions", "compat": { "supportsUsageInStreaming": false, "maxTokensField": "max_tokens" }, "models": [...] } } } ``` | Field | Description | | --------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | `supportsStore` | Provider supports `store` field | | `supportsDeveloperRole` | Use `developer` vs `system` role | | `supportsReasoningEffort` | Support for `reasoning_effort` parameter | | `supportsUsageInStreaming` | Supports `stream_options: { include_usage: true }` (default: `true`) | | `maxTokensField` | Use `max_completion_tokens` or `max_tokens` | | `requiresToolResultName` | Include `name` on tool result messages | | `requiresAssistantAfterToolResult` | Insert an assistant message before a user message after tool results | | `requiresThinkingAsText` | Convert thinking blocks to plain text | | `requiresReasoningContentOnAssistantMessages` | Include empty `reasoning_content` on all replayed assistant messages when reasoning is enabled | | `thinkingFormat` | Use `reasoning_effort`, `openrouter`, `deepseek`, `together`, `zai`, `qwen`, `chat-template`, or `qwen-chat-template` thinking parameters | | `chatTemplateKwargs` | `chat_template_kwargs` values for `thinkingFormat: "chat-template"`; use `{ "$var": "thinking.enabled" }` or `{ "$var": "thinking.effort" }` for Atomic-controlled thinking values | | `cacheControlFormat` | Use Anthropic-style `cache_control` markers on the system prompt, last tool definition, and last user/assistant text content. Currently only `anthropic` is supported. | | `supportsStrictMode` | OpenAI-compatible strict JSON-schema function tools. This is not a general guarantee for every API. | | `supportsStrictTools` | Anthropic/Bedrock strict-tool capability, normally generated from verified model metadata. | | `supportsOpenAIGrammarTools` | Canonical Pi capability for OpenAI Lark/regex custom tools. Keep false unless the endpoint passes custom tools through unchanged. | | `supportsGrammarTools` | Atomic compatibility alias for `supportsOpenAIGrammarTools`; the canonical field wins if both disagree. | | `supportsLongCacheRetention` | Whether the provider accepts long cache retention when cache retention is `long`: `prompt_cache_retention: "24h"` for OpenAI prompt caching, or `cache_control.ttl: "1h"` when `cacheControlFormat` is `anthropic`. Default: `true`. | | `openRouterRouting` | OpenRouter provider routing preferences. This object is sent as-is in the `provider` field of the [OpenRouter API request](https://openrouter.ai/docs/guides/routing/provider-selection). | | `vercelGatewayRouting` | Vercel AI Gateway routing config for provider selection (`only`, `order`) | ### Constrained tool sampling Tools may request `{ type: "json_schema", strict: "prefer" | "require" }` or `{ type: "grammar", variants: { openai_lark?: string, openai_regex?: string } }`. `prefer` may fall back to ordinary tool calling; `require` must fail if the active provider/model cannot enforce the schema. Grammar tools use OpenAI custom-tool syntax and fall back to normal function handling when grammar capability is absent. Do not infer support from a provider name: Atomic carries the model's explicit capability metadata through built-in catalogs, dynamic catalogs, overrides, SDK/RPC model objects, and isolated execution. Strict JSON-schema support currently includes OpenAI, Anthropic, capable Bedrock Converse models, Mistral, and Gemini 3 through Google/Vertex. Earlier Gemini models cannot enforce required parameters: `prefer` falls back and `require` fails. OpenAI grammar tools are limited to capable GPT-5+ models on endpoints known to preserve custom tools; gateways such as OpenRouter may normalize and break them. ### Catalog freshness and precedence Authenticated remote catalogs are cached in `models-store.json`. Atomic revalidates pi.dev catalogs with the stored ETag through `If-None-Match`; an empty `304 Not Modified` is success and retains the cached body while updating its check time. A newer bundled catalog wins over an older persisted overlay even when package file mtimes are misleading. Final visibility is built-ins, persisted/remote data subject to freshness, configured `.pi` then `.atomic` layers, and live provider catalogs/overrides. Provider failures retain the last usable provider-specific snapshot. Claude Opus 5 is present in the generated Anthropic and Amazon Bedrock catalogs. Its metadata enables adaptive thinking, including `xhigh` where advertised. Bedrock uses its generated inference-profile ID, prompt-caching and strict-tool metadata, and preserves provider/AWS validation errors. Custom entries must reproduce those capabilities honestly rather than copying a display name alone. `openrouter` uses `reasoning: { effort }`. `together` uses `reasoning: { enabled }` and also `reasoning_effort` when `supportsReasoningEffort` is enabled. `qwen` uses top-level `enable_thinking`. Use `qwen-chat-template` for local Qwen-compatible servers that require `chat_template_kwargs.enable_thinking` and `preserve_thinking`. Use `chat-template` for vLLM/Hugging Face chat templates that need configurable `chat_template_kwargs`, such as `chatTemplateKwargs: { "thinking": { "$var": "thinking.enabled" } }` for DeepSeek V3.x templates. `cacheControlFormat: "anthropic"` is for OpenAI-compatible providers that expose Anthropic-style prompt caching through `cache_control` markers on text content and tool definitions. Example: ```json theme={null} { "providers": { "openrouter": { "baseUrl": "https://openrouter.ai/api/v1", "apiKey": "$OPENROUTER_API_KEY", "api": "openai-completions", "models": [ { "id": "openrouter/anthropic/claude-3.5-sonnet", "name": "OpenRouter Claude 3.5 Sonnet", "compat": { "openRouterRouting": { "allow_fallbacks": true, "require_parameters": false, "data_collection": "deny", "zdr": true, "enforce_distillable_text": false, "order": ["anthropic", "amazon-bedrock", "google-vertex"], "only": ["anthropic", "amazon-bedrock"], "ignore": ["gmicloud", "friendli"], "quantizations": ["fp16", "bf16"], "sort": { "by": "price", "partition": "model" }, "max_price": { "prompt": 10, "completion": 20 }, "preferred_min_throughput": { "p50": 100, "p90": 50 }, "preferred_max_latency": { "p50": 1, "p90": 3, "p99": 5 } } } } ] } } } ``` Vercel AI Gateway example: ```json theme={null} { "providers": { "vercel-ai-gateway": { "baseUrl": "https://ai-gateway.vercel.sh/v1", "apiKey": "$AI_GATEWAY_API_KEY", "api": "openai-completions", "models": [ { "id": "moonshotai/kimi-k2.5", "name": "Kimi K2.5 (Fireworks via Vercel)", "reasoning": true, "input": ["text", "image"], "cost": { "input": 0.6, "output": 3, "cacheRead": 0, "cacheWrite": 0 }, "contextWindow": 262144, "maxTokens": 262144, "compat": { "vercelGatewayRouting": { "only": ["fireworks", "novita"], "order": ["fireworks", "novita"] } } } ] } } } ```