LLM Models

Every agent needs a model. Models are not integrations: they are configured separately, through model providers. This page covers how a provider is connected, how a model is created from it, and which model an agent ends up using.

The three layers

Providers

The providers available on your instance depend on the MODULES setting (see Environment Variables). The registry endpoint always returns the exact list.

API key providers (core-providers, loaded by default)

The two custom endpoint providers are the escape hatch: give them a base_url_override to reach a self-hosted or third-party server (Ollama, vLLM, LiteLLM and similar). The API-key providers all use one of the four wire formats the platform's LLM proxy serves, so inference reaches the vendor through the same sidecar path whichever one you pick.

Google, Vertex, Azure OpenAI and AWS Bedrock are not native providers. To use a model served by one of them, put a gateway that exposes an OpenAI or Anthropic compatible API in front of it and connect that gateway through a custom endpoint provider.

Subscription providers (opt-in modules)

Two modules let an organization run agents on a personal subscription instead of an API key. Neither is in the MODULES default: an operator adds them deliberately.

These use OAuth, not a pasted key. The real token never enters the agent container: the agent holds a placeholder and the sidecar swaps in the real bearer server-side. Subscription runs need an isolating run adapter (docker or firecracker), because the plain process adapter does not deliver subscription credentials.

Connect a subscription

1. Add the module to MODULES and restart the platform. Keep the rest of your list:

   MODULES=oidc,webhooks,mcp,core-providers,@appstrate/module-chat,@appstrate/module-claude-code,@appstrate/module-codex

   Load one or both. Without the module the provider id is unknown to the platform: no credential can be created for it, and no subscription traffic is possible.

2. Start the connection from the dashboard. In the organization settings, open the Models page, then the Model Provider Keys tab, click Add key and pick the provider. The dashboard mints a pairing token, valid for five minutes, and shows a Connect via CLI command: npx @appstrate/connect-helper@<version> <token>, pinned to the helper version your platform speaks. Copy it and run it in a terminal on your own machine, where a browser can open. The login runs there because the Claude Code authorize endpoint only accepts loopback redirect addresses.

3. Sign in when the helper asks. The helper runs the browser login against the provider and sends the credentials back to Appstrate. The dashboard shows "Waiting for the connection" until it arrives. Closing the dialog does not cancel a connection already in progress, and an expired command can be replaced with Generate a new command.

4. Pick models. Connecting from the Model Provider Keys tab only creates the credential. Only the first-run onboarding flow seeds the provider's featured models. Open the Models tab, choose Add model and pick the new key under My keys.

When the stored subscription token can no longer be used, the credential shows Reconnection required with a Reconnect action, and the models bound to it are flagged until you reconnect.

Models are the provider's offer, not discovered per credential

Appstrate never calls the vendor to list the models of a subscription credential. The models you can choose are the provider's offer: the records of Pi's pinned model catalogue for that provider (anthropic for claude-code, openai-codex for codex). They are the same for every credential, and nothing is stored per credential. POST /api/model-provider-credentials/discover refuses these providers with a 400. A model that your plan does not serve is therefore still listed, and fails at the first run (Pro, Max and Team differ for Claude).

The same rule covers testing. POST /api/models/{id}/test runs an offline check on the stored token: whether it is well-formed and unexpired. It is not a signature check and not a live call, so a passing test does not prove the token still works upstream. A subscription credential is proven live only when the first agent run presents it to the vendor.

Compliance

Using a personal subscription to power a product is a policy question that belongs to the operator, and vendor terms can change. For production and team automation, use an API key: both vendors' clean contract is API-key billing. The reasoning is in SUBSCRIPTION_COMPLIANCE.md, which says plainly that Appstrate does not certify compliance with any vendor's terms, and records the following:

• What the code guarantees. Pi, the agent engine, builds every subscription request. The platform forges no client identity, the sidecar swaps the bearer and forwards every other header Pi signed unchanged, and the real token stays server-side. The token is the user's or organization's own and is never pooled across tenants. The platform sends no request that a subscription token authenticates for testing or model discovery. The one honest narrowing: the upstream request is made by the sidecar, not by a vendor binary, so Appstrate does not claim transport-level client identity.
• Claude Code. The file records a timeline of policy changes through 2026, and quotes Anthropic's Agent SDK documentation as not allowing third-party developers to offer claude.ai login or rate limits for their products without prior approval. It concludes that pointing Appstrate at a personal Claude subscription is an operator-owned grey-zone choice, not a sanctioned integration.
• Codex. OpenAI had not banned subscription OAuth in third-party tools when the file was reviewed, but has not endorsed it either. The path relies on the Codex OAuth client and could be closed at any time.
• Re-verify before relying on it. The file was last reviewed on 2026-07-08, calls the policy half volatile, and ends with a checklist of vendor terms to re-check. A burst of 401 or 410 answers on a subscription credential is its operational sign that a vendor has closed the path.

Adding a provider and a model

Create the credential first, then the model that points at it.

# 1. Store the API key (encrypted at rest)
appstrate api POST /api/model-provider-credentials -d '{
  "providerId": "openai",
  "api_key": "sk-..."
}'
# -> { "id": "<credentialId>", ... }

# 2. Create a model bound to it
appstrate api POST /api/models -d '{
  "modelId": "<id from the provider offer>",
  "credentialId": "<credentialId>"
}'

• label is optional on both calls. The platform derives one and avoids duplicates.
• For a custom endpoint provider, add "base_url_override": "https://llm.internal.example.com/v1" to the credential.
• A named provider only accepts model ids from its own offer (model_not_offered otherwise). The offer is the provider's records in the model registry bundled with your platform version, plus the newer ones the instance has read from the live model catalog. Gateways (OpenAI-compatible, Anthropic-compatible) and OpenRouter accept any id.
• POST /api/model-provider-credentials/test and POST /api/model-provider-credentials/{id}/test check a key before or after saving. POST /api/models/{id}/test checks one model.
• POST /api/model-provider-credentials/discover lists the models an endpoint serves.
• POST /api/models/seed creates several models from the provider's offer for one credential in a single call.
• For OpenRouter, GET /api/models/openrouter?q=claude searches its catalogue. The registry marks such a provider with live_model_search: true: its models are searched live, and any model id is accepted.
• Each model returned by GET /api/models carries pi_provider, the key of the Pi registry provider it is served through (for example moonshotai for moonshot). It is null for a gateway and for an aliased system model.
• Per-model overrides (contextWindow, maxTokens, reasoning, input, cost) are optional. Left out, they come from the model catalogue and follow catalogue updates.

For a subscription provider, start from the dashboard so the pairing token and command are generated for you.

The dashboard also seeds a provider's featured models the first time you connect it.

Where the offer comes from

A provider's offer comes from the model registry that ships with your platform version, the one the agent engine (Pi) uses. Two things change what you see in it:

• A platform upgrade can change the offer. A model the registry no longer records can no longer be added. A model you already added on it stays in your list with the values stored, but it loses the catalog defaults (label, limits, capabilities, price) and runs unpriced unless you set a cost on it. If the operator listed such a model in SYSTEM_PROVIDER_KEYS, the platform refuses to boot until it is removed, and bun run verify:system-models run before the deploy catches it. The release notes of each version say which providers lost models.
• A live catalog adds new models between releases. Each API process reads a signed file from get.appstrate.dev when it starts and every hour, and holds it in memory. It lists models a newer registry records and your build can already serve, so a new vendor model becomes selectable without a platform release. It only adds: it never changes a model of the bundled registry, a system model or a featured model. The read sends nothing about your instance. Until a process has read the file (just after a restart, or while the channel is unreachable), a model added from it runs without its catalog defaults. An operator turns the read off with MODEL_CATALOG_URL=off, which leaves the bundled registry alone (see Environment Variables). The design is in MODEL_CATALOG.md.

System models and organization models

An operator can offer models to every organization with the SYSTEM_PROVIDER_KEYS environment variable: a JSON array of provider credentials, each with the models it exposes. System models appear in GET /api/models next to the organization's own. You cannot bind your own models to a system credential: to add models for your organization, create your own credential.

Model aliases

An operator can expose a system model under a vanity name (for example appstrate-medium) while the real model and endpoint stay on the server. Set "aliased": true and an explicit "label" on the entry. API callers see the alias, not the backing. This hides what the platform hands out, not what the model reveals about itself when asked. Subscription providers cannot back an alias. The pattern and its limits are described in MODEL_ALIASES.md.

Which model does a run use

The first match wins:

1. The model named on the run request (modelId), or on a schedule.
2. The model set on the agent.
3. The organization default.
4. The system default (the system model flagged as default).

If none is configured, the run cannot start: the launch answers 400 model_not_configured and creates no run.

A disabled model cannot be the organization default, and a model whose credential needs reconnection cannot either: PUT /api/models/default answers 409 with model_disabled or model_needs_reconnection. Disabling the current default (PATCH /api/models/{id} with enabled: false) is refused the same way (model_disabled): move or clear the default first. Two requests that would do both at once cannot both succeed, one of them gets the 409.

# Organization default (send null to clear)
appstrate api PUT /api/models/default -d '{"modelId": "<modelId>"}'

# Per-agent model
appstrate api PATCH /api/agents/@acme/triage/model -d '{"modelId": "<modelId>"}'

GET /api/agents/{scope}/{name}/model returns the current setting. Changing an agent's model needs the agents:configure permission. The modelId in these calls is the Appstrate model id, not the vendor's.

Generation settings: temperature and reasoning

Two request settings can be tuned per agent or per run, without changing the model itself:

• Temperature, from 0 (deterministic) to 1 (more varied).
• Reasoning level, one of off, minimal, low, medium, high, xhigh and max.

Both are optional. Left unset they inherit, and an unset reasoning level resolves to medium (or, when the model does not take medium, the nearest level it supports, looking upward first).

Where to set them

• On an agent. Open the agent, then the Default settings tab (it needs the agents:configure permission). The Models card has the model select, a Temperature slider, a Reasoning level selector and a Save model settings button. The values apply to every run of the agent in that space.
• For one run or one schedule. The Run with options dialog and the schedule form carry the same two controls. Empty fields inherit the agent's defaults.
• In the chat. The chat's model picker ("Model and generation") has the same controls.
• Through the API. generation on PATCH /api/agents/{scope}/{name}/model and on the run request, and generation_config_override on schedules, each with temperature and reasoning_level. See Agents and Runs.

The model form on the Models page of the organization settings only declares whether a model reasons, with a Reasoning checkbox among its capabilities. It has no temperature or reasoning level control.

What each model accepts

Each model declares which of these settings it supports. GET /api/models returns that as a generation object on every model, and the controls follow it:

• The slider's first stop is Auto ("Provider default"), then 0 to 1 in steps of 0.1.
• When a model declares a setting unsupported, its control is disabled, greyed out and badged Unsupported, with the note "The selected model declares this setting unsupported."
• Reasoning levels the model does not confirm are disabled one by one, so you only see selectable levels. A model with no selectable level shows the whole reasoning control as unsupported.
• Some models cannot combine a custom temperature with reasoning. For those, a temperature is not kept alongside a reasoning level other than off.
• When you switch the model in the form, any value the new model refuses is cleared instead of being kept.

The capabilities come from the model catalogue, so they follow catalogue updates. A model the catalogue has no record of, such as a model on an OpenAI-compatible or Anthropic-compatible gateway, has unknown temperature support, so the temperature stays available. Its reasoning levels follow the Reasoning checkbox you set on the model: a model declared as reasoning takes off, low, medium and high, any other takes off only. minimal, xhigh and max stay refused on such a model, and the default level stays medium. On an OpenAI-compatible model, off sends no reasoning parameter at all, so the server keeps its own behaviour and some models still reason: the level control labels it "Not sent: the server decides" and says so in its hint. On an Anthropic-compatible model, off disables thinking.

When a setting is refused

What happens depends on where the value comes from:

• Saving it. Storing a setting the model refuses, on an agent, a space or a schedule, answers 400 invalid_request, with param naming the field.
• Sending it with a run request. A generation value in the request itself that the model refuses answers 400, with one of the codes temperature_unsupported, reasoning_unsupported, reasoning_level_unsupported or temperature_with_reasoning_unsupported. A value you set for one run is never silently changed.
• A stored value the model now refuses. If the agent's defaults or a schedule's override were valid when saved but the model that runs now refuses them (you changed the default model, or the catalogue changed), the run proceeds without that setting. A scheduled fire has nobody to refuse it, so the run is not blocked.

In the last case the run's log says so. For each dropped setting the run gets one warn entry named generation_setting_dropped, with the setting, its value, the model and reason: "refused_by_model". Its message names the setting, its value and the model, and says the setting is ignored for this run. The platform's own log also records Stored generation settings refused by the model, dropped for this run. A request value that sets the same field takes precedence over the stored one, so that field is not reported as dropped.

Cost tracking

Appstrate records token usage for every inference call in a ledger and prices it from the model's rate card: the model catalogue, OpenRouter's price list for OpenRouter models, or the cost you set on the model. Prices are in US dollars per million tokens, and a run's cost is in USD, the sum of its ledger rows. The mechanics are in RUN_COST.md. Usage on a subscription provider goes through the same ledger.