Models

Connect LLM providers, secure upstream credentials, and publish governed model endpoints

A model (model connection) is one upstream LLM endpoint agents call through a Gateway. Dome routes the request, injects the provider credential, applies Rules and Guards, and records the result.

Overview

A model connection represents one upstream model in Dome. The connection stores the configuration and credentials Dome needs to call that model. An agent selects the connection by putting its name in the request's model field.

A pool groups multiple models behind one name. Dome selects one member for each request. Prefer a pool when you need routing, traffic distribution, or failover. Refer to Pools concept.

The typical workflow is:

Choose a provider and model identifier.
Add the model with its upstream credentials.
Attach the model directly to a Gateway or add it to a model pool.
Allow the Gateway on the agent.
Verify the route with that agent's API key.

Providers

The provider determines how Dome translates requests and where Dome places the upstream credential. You cannot change the provider after adding the model.

Provider IDs fall into three groups.

Native providers. Dome translates between the Gateway request and the provider's native wire format: openai, anthropic, google, azure_openai, and bedrock.
Hosted OpenAI-compatible providers. Dome sends requests through the provider's OpenAI-compatible API (for example mistral, groq, together, moonshot, openrouter, databricks).
Bring your own endpoint. Use openai_compatible for an OpenAI-shaped endpoint such as vLLM or Ollama. Use custom when Dome should forward provider-specific requests without translating them.

Dome supplies a base URL for hosted providers that have a default. For Google, Azure OpenAI, Bedrock, Databricks, openai_compatible, and custom, you must provide an endpoint or the provider-specific details needed to locate the model. The full provider ID list is on the Models reference.

Discover valid provider and model ids from the same catalog the dashboard picker uses: dome models providers or dome_models_providers. Curated lists exist for providers with stable public ids (OpenAI, Anthropic, Google) and always include a default. Other providers return an empty model list and accept any model string the upstream accepts. A model your upstream supports but the catalog does not list is still valid — pass it as --model <id>.

Credentials

Agent API keys authenticate inbound requests to Dome. Model credentials authenticate Dome's outbound requests to the provider, and agents never receive them.

Authentication method	Credential type	What Dome sends
`none`	`none`	No provider credential
`api-key`	`shared`	One workspace-managed API key for every call
`api-key`	`per-user`	An API key linked to the verified end user
`oauth`	`shared`	One administrator-authorized OAuth grant for every call
`oauth`	`per-user`	An OAuth grant linked to the verified end user

When the provider requires no authentication, set both the authentication method and credential type to none. When the provider uses an API key or OAuth, the credential type must be shared or per-user. If you omit both settings, Dome uses api-key with shared.

With per-user credentials, Dome uses the verified act-as identity on each request to select the provider credential. If that end user has not connected a credential, Dome returns HTTP 401 with a short-lived provision_url. The end user opens the URL to enter an API key or complete OAuth consent, then retries the request. Refer to Errors and denials for runtime handling.

Google models with shared credentials use a service account. On a self-hosted gateway, Google and Bedrock models can instead use the gateway's cloud-native identity. A Dome-hosted gateway cannot assume a role or workload identity from your cloud account.

Attributes

Attributes add trusted metadata to a model for rules to evaluate. For example, a rule can allow models with region=eu for a European workload or deny models whose data_class does not match the request. Attributes do not change how Dome calls the provider.

You can set attributes when you add a model or change them later.

Filter window

A model can change how much streamed output Dome buffers before applying outbound Guards. Dome uses the largest window configured on the workspace, model, or request. A model can therefore increase inspection coverage but cannot reduce the workspace minimum. A value of 0 uses the workspace setting. A model can buffer at most 1 MiB or 4,096 tokens.

You can set a filter window when you add a model or change it later.

Requirements

Before you begin:

Authenticate to Dome and select a workspace
Have the provider, model identifier, and provider-specific configuration
Have a provider credential unless the model uses no authentication or workload identity
Have a Gateway ready when you want the model reachable

Permissions

Model connection operations require platform permissions. Each operation states its required permission inline.

Default roles	Permission	Grants
All workspace roles	`gateways.view`	List and inspect models
`admin`, `operator`	`gateways.manage`	Add, update, and remove models

Add model

Add a model for one provider endpoint. You can attach the model directly to a Gateway during this operation. If you do not attach it, agents cannot call the model until you attach it later.

Requires gateways.manage.

dome models add claude-prod \
  --provider anthropic \
  --model claude-sonnet-4-6 \
  --api-key "$ANTHROPIC_API_KEY" \
  --gateway prod-llms

Add --endpoint or --provider-config when the provider requires more configuration.

Reference: dome models add

Tool: dome_models_add

{
  "name": "claude-prod",
  "provider": "anthropic",
  "provider_config": {
    "model": "claude-sonnet-4-6"
  },
  "api_key": "{{ANTHROPIC_API_KEY}}",
  "gateways": ["prod-llms"]
}

Reference: dome_models_add

POST /v1/models
Content-Type: application/json

{
  "workspace_id": "{{WORKSPACE_ID}}",
  "name": "claude-prod",
  "provider": "anthropic",
  "provider_config": {
    "model": "claude-sonnet-4-6"
  },
  "auth_method": "api-key",
  "credential_type": "shared",
  "secret_values": {
    "x-api-key": "{{ANTHROPIC_API_KEY}}"
  }
}

The endpoint returns the model's UUID. The other management endpoints use this UUID to identify the model. Attach the model to a Gateway in a separate operation.

Reference: CreateLLMModelConnection

Connect an LLM model

Add an Anthropic model named "claude-prod" for claude-sonnet-4-6 using my API key. Attach it to the "prod-llms" Gateway.

Agents cannot call a model until you attach it directly to a Gateway or add it to a pool attached to one.

Update model

Update a model to change its name, model identifier, endpoint, authentication, attributes, or filter window. You cannot change the provider after creating the model. To switch providers, add a new model and replace the old connection in any pools or Gateways.

Requires gateways.manage.

dome models update claude-prod \
  --model claude-sonnet-4-6

Only the flags you pass change. --provider-config replaces the full provider configuration instead of merging with it.

Reference: dome models update

Tool: dome_models_update

{
  "name": "claude-prod",
  "provider_config": {
    "model": "claude-sonnet-4-6"
  }
}

Only the parameters you send change. provider_config and attributes each replace the existing object.

Reference: dome_models_update

PATCH /v1/models/{{CONNECTION_ID}}
Content-Type: application/json

{
  "workspace_id": "{{WORKSPACE_ID}}",
  "provider_config": {
    "model": "claude-sonnet-4-6"
  }
}

Omitted fields keep their stored values. When present, provider_config replaces the full object. To replace or clear attributes, headers, credentials, or the filter window, send the new value and set its corresponding *_provided field to true.

Reference: UpdateLLMModelConnection

Update a model

Update the "claude-prod" model to use claude-sonnet-4-6. Keep its provider, credentials, attributes, and Gateway membership unchanged.

Change credentials

You can change the authentication method, credential type, or shared API key. Switching authentication methods deletes the previously stored provider credential.

dome models update claude-prod \
  --auth-method api-key \
  --credential-type per-user

The current CLI can change the authentication method or credential type, but it does not replace an existing shared API key. Use MCP or the API to rotate that key.

Reference: dome models update

Tool: dome_models_update

{
  "name": "claude-prod",
  "auth_method": "api-key",
  "credential_type": "shared",
  "api_key": "{{ANTHROPIC_API_KEY}}"
}

Reference: dome_models_update

PATCH /v1/models/{{CONNECTION_ID}}
Content-Type: application/json

{
  "workspace_id": "{{WORKSPACE_ID}}",
  "auth_method": "api-key",
  "credential_type": "shared",
  "secret_values": {
    "x-api-key": "{{ANTHROPIC_API_KEY}}"
  },
  "secret_values_provided": true
}

Reference: UpdateLLMModelConnection

Rotate a model credential

Replace the shared API key on "claude-prod" with my new Anthropic API key.

Change attributes

You can replace the trusted attributes that rules evaluate for this model. Send an empty object to clear them.

dome models update claude-prod \
  --attributes '{"region":"eu","data_class":"restricted"}'

Reference: dome models update

Tool: dome_models_update

{
  "name": "claude-prod",
  "attributes": {
    "region": "eu",
    "data_class": "restricted"
  }
}

Reference: dome_models_update

PATCH /v1/models/{{CONNECTION_ID}}
Content-Type: application/json

{
  "workspace_id": "{{WORKSPACE_ID}}",
  "attributes": {
    "region": "eu",
    "data_class": "restricted"
  },
  "attributes_provided": true
}

Reference: UpdateLLMModelConnection

Change model attributes

Replace the attributes on "claude-prod" with region "eu" and data class "restricted".

Change filter window

You can change how much streamed output Dome buffers for outbound Guard inspection. A model can buffer at most 1 MiB or 4,096 tokens. Set both values to 0 to use the workspace minimum.

dome models update claude-prod \
  --filter-window-bytes 65536 \
  --filter-window-tokens 1024

Reference: dome models update

PATCH /v1/models/{{CONNECTION_ID}}
Content-Type: application/json

{
  "workspace_id": "{{WORKSPACE_ID}}",
  "filter_window_bytes": 65536,
  "filter_window_tokens": 1024,
  "filter_window_provided": true
}

Reference: UpdateLLMModelConnection

Get model

Retrieve a model to inspect its provider configuration, authentication, attributes, filter window, and Gateway membership. Use this information before an update or when troubleshooting routing.

Requires gateways.view.

dome models get claude-prod

Reference: dome models get

Tool: dome_models_get

{
  "name": "claude-prod"
}

Reference: dome_models_get

GET /v1/models/{{CONNECTION_ID}}?workspace_id={{WORKSPACE_ID}}

The management API addresses a model by UUID. If you know only its name, call ListLLMModelConnections to find the corresponding ID.

Reference: GetLLMModelConnection

Get a model

Get the "claude-prod" model and confirm its provider, model identifier, authentication method, and Gateway membership.

List models

Retrieve the models in a workspace with their providers and configurations. Use the list to find a model to inspect, update, add to a pool, or remove.

Requires gateways.view.

dome models list

Reference: dome models list

Tool: dome_models_list

{}

Reference: dome_models_list

GET /v1/models?workspace_id={{WORKSPACE_ID}}

Reference: ListLLMModelConnections

List models

List the models in this workspace with their providers and model identifiers.

Remove model

Remove a model to delete its connection and provider credentials from the workspace.

Requires gateways.manage.

Removing a model also removes it from model pools and Gateways. Before deleting a model that serves traffic, replace it in those routes or move the traffic elsewhere.

dome models rm claude-prod

Reference: dome models rm

Tool: dome_models_remove

{
  "name": "claude-prod"
}

Reference: dome_models_remove

DELETE /v1/models/{{CONNECTION_ID}}?workspace_id={{WORKSPACE_ID}}

Reference: DeleteLLMModelConnection

Remove a model

Remove the "claude-prod" model after confirming no pools or Gateways still depend on it.

Attach to a Gateway

Attach a model directly to a Gateway when its name should always route to that one connection. The attachment makes the model reachable at the Gateway endpoint but does not authorize an agent to call it. The agent must allow the Gateway, and rules must permit the request.

When a route needs failover or weighted distribution across multiple models, add the model to a model pool and attach the pool instead.

Requires gateways.manage.

dome gateways models add prod-llms claude-prod

Reference: dome gateways models add

Tool: dome_gateways_model_add

{
  "gateway": "prod-llms",
  "model": "claude-prod"
}

Reference: dome_gateways_model_add

Publish a direct model

Add the "claude-prod" model directly to the "prod-llms" Gateway.

Verify model

Call the model through the Gateway with an agent API key. A successful response confirms the model is reachable, rules permit the request, and the upstream accepted the configured credential.

OpenAI-compatible client

from openai import OpenAI

client = OpenAI(
    api_key="{{DOME_AGENT_KEY}}",
    base_url="https://{{GATEWAY_HOST}}/gateways/{{GATEWAY_ID}}/v1",
)

response = client.chat.completions.create(
    model="claude-prod",
    messages=[{"role": "user", "content": "Return OK."}],
)

If the model is missing or denied, check Gateway membership, the agent's allowed resources, and rules. If the upstream rejects the call, check the model's provider configuration and credentials.

Manage OAuth

With shared OAuth, an administrator completes consent once and Dome uses the resulting provider tokens for every call. With per-user OAuth, each verified end user completes consent separately. When an end user has not connected OAuth, the first request returns a provision_url where the user can complete consent.

Configure the OAuth client when you add or update the model. If the provider publishes OAuth metadata, Dome can discover the required endpoints. Otherwise, you must provide the authorization, token, revocation, and registration URLs. A manually registered client requires a client ID and secret. With dynamic client registration, Dome obtains them from the provider.

Connect OAuth

Connect OAuth after adding a model configured to use shared OAuth. Dome returns a one-time URL that remains valid for about ten minutes. Open the URL to complete consent at the provider.

Requires gateways.manage.

dome models oauth-connect claude-prod

Reference: dome models oauth-connect

Tool: dome_models_oauth_connect

{
  "name": "claude-prod"
}

Reference: dome_models_oauth_connect

POST /v1/models/{{CONNECTION_ID}}/oauth/connect
Content-Type: application/json

{
  "workspace_id": "{{WORKSPACE_ID}}"
}

Reference: BeginLLMOAuthConnect

Connect model OAuth

Start shared OAuth consent for the "claude-prod" model.

Disconnect OAuth

Disconnect shared OAuth to revoke the stored provider tokens. The OAuth client configuration remains, so you can reconnect without configuring the client again. Disconnecting an already disconnected model has no effect.

Requires gateways.manage.

dome models oauth-disconnect claude-prod

Reference: dome models oauth-disconnect

Tool: dome_models_oauth_disconnect

{
  "name": "claude-prod"
}

Reference: dome_models_oauth_disconnect

POST /v1/models/{{CONNECTION_ID}}/oauth/disconnect
Content-Type: application/json

{
  "workspace_id": "{{WORKSPACE_ID}}"
}

Reference: DisconnectLLMOAuth

Disconnect model OAuth

Disconnect shared OAuth credentials from the "claude-prod" model.

Next steps

Models concept for how connections and attributes work and Models reference for providers and credentials
Create a model pool for routing, traffic distribution, and failover
Manage membership and grants in Gateways
Authenticate clients and route model traffic in Develop

Models

On this page