LLM Models - Dome

Model tools register LLM upstreams the gateway proxies — Anthropic, OpenAI, Azure OpenAI, Google, Bedrock, hosted OpenAI-compatible vendors (Groq, Mistral, Together, Fireworks, DeepSeek, xAI, Perplexity, Cohere, OpenRouter, Cerebras, NVIDIA, DeepInfra, SambaNova, AI21, Databricks), and self-hosted endpoints — and group them into routing pools. Pools give agents one logical name that fans out across providers with priority, weighting, and failover. Models share workspace scope, gateway placement, and credential handling with Tools. Use models for inference endpoints; use tools for MCP servers.

Model CRUD

dome_model_list

List LLM model connections in the active workspace.

Parameter	Type	Required	Description
—	—	—	No parameters

Permission: gateway.view

dome_model_get

Get an LLM model connection by name.

Parameter	Type	Required	Description
`name`	string	Yes	Model name

Permission: gateway.view

dome_model_add

Add an LLM model connection.

Parameter	Type	Required	Description
`name`	string	Yes	Connection name (unique within workspace)
`provider`	string	Yes	`openai`, `anthropic`, `google`, `bedrock`, `azure_openai`, `mistral`, `groq`, `together`, `fireworks`, `deepseek`, `xai`, `perplexity`, `cohere`, `openrouter`, `cerebras`, `nvidia`, `deepinfra`, `sambanova`, `ai21`, `databricks`, `openai_compatible`, `custom`. Refer to Supported providers reference.
`provider_config`	object	No	Provider-specific config. Common keys: `model`, `endpoint` (omit `endpoint` for hosted providers to use the registry’s default base URL)
`auth_method`	string	No	`none`, `api-key`, `oauth` (default: `api-key`)
`credential_type`	string	No	`none`, `shared`, `per-user` (default: `shared`)
`api_key`	string	No	API key for `api-key`+`shared`. Stored under the provider’s managed auth header (`Authorization` for OpenAI/Bedrock, `x-api-key` for Anthropic, `api-key` for Azure, `x-goog-api-key` for Google)
`secret_values`	object	No	Advanced: per-connection credential bundle for custom secret-source headers
`attributes`	object	No	Cedar attributes used by authorization rules
`gateways`	array	No	Gateway names to attach the model to directly at create time. Without it the model is unreachable until attached (`dome_gateway_model_add`).

Permission: gateway.manage

Add an Anthropic model

{
  "name": "dome_model_add",
  "arguments": {
    "name": "claude-sonnet",
    "provider": "anthropic",
    "provider_config": { "model": "claude-3-5-sonnet-20241022" },
    "api_key": "sk-ant-..."
  }
}

dome_model_update

Update an LLM model connection. Only provided fields change.

Parameter	Type	Required	Description
`name`	string	Yes	Model name
`new_name`	string	No	Rename the connection
`provider_config`	object	No	Replace `provider_config`
`auth_method`	string	No	New auth method
`credential_type`	string	No	New credential scope
`api_key`	string	No	Rotate the shared API key. Stored under the provider’s managed auth header (`Authorization` for OpenAI/Bedrock, `x-api-key` for Anthropic, `api-key` for Azure, `x-goog-api-key` for Google)
`attributes`	object	No	Replace Cedar attributes (pass `{}` to clear)

Permission: gateway.manage Provider is immutable after create — to switch providers, add a new connection.

dome_model_remove

Remove an LLM model connection. Marked destructive.

Parameter	Type	Required	Description
`name`	string	Yes	Model name

Permission: gateway.manage

dome_model_oauth_connect

Begin the shared-OAuth admin consent flow for a model configured with auth_method=oauth, credential_type=shared (Anthropic OAuth, Azure AAD service principal, Google Workload Identity for Vertex). Returns a one-shot URL valid for ~10 minutes.

Parameter	Type	Required	Description
`name`	string	Yes	Model name

Permission: gateway.manage

dome_model_oauth_disconnect

Disconnect (revoke) shared-OAuth credentials for a model. Idempotent. Marked destructive.

Parameter	Type	Required	Description
`name`	string	Yes	Model name

Permission: gateway.manage

Pool CRUD

Pools are logical names agents call. The gateway picks a member at request time using the pool’s routing strategy and falls back across priority buckets when a member fails.

dome_model_pool_list

List LLM pools in the active workspace. Permission: gateway.view

dome_model_pool_get

Get a pool with its members.

Parameter	Type	Required	Description
`name`	string	Yes	Pool name

Permission: gateway.view

dome_model_pool_create

Create an LLM pool.

Parameter	Type	Required	Description
`name`	string	Yes	Pool name (unique within workspace)
`description`	string	No	Free-form description
`match_when`	object	No	Predicate selecting this pool by request shape (e.g. `{"prompt_tokens":{"gt":100000}}`). Refer to match_when predicates reference for the full grammar — operators (`gt`/`lt`/`eq`, `in`, `prefix`, `suffix`, `regex`) and composition (`any`, `all`, `not`).
`is_default`	bool	No	Mark as the workspace default pool
`cache_ttl_secs`	int32	No	Exact-match cache TTL (`0` disables cache)
`cache_scope`	string	No	`workspace` or `caller` (default: `workspace`)
`routing_strategy`	string	No	`priority_weighted`, `round_robin`, `least_loaded` (default: `priority_weighted`)
`strategy_scope`	string	No	Scope of stateful routing counters: `workspace` (default) or `caller`. Ignored by `priority_weighted`.
`failover_max`	string	No	Failover ceiling. `"all"` walks every member; integer `N` gives primary + N retries (`0` disables failover). Defaults to `"all"`.
`gateways`	array	No	Gateway names to attach the pool to at create time. Without it the pool is unreachable until attached (`dome_gateway_model_pool_add`).

Permission: gateway.manage

dome_model_pool_update

Update a pool. Only provided fields change.

Parameter	Type	Required	Description
`name`	string	Yes	Pool name
`new_name`	string	No	Rename
`description`	string	No	Update description
`match_when`	object	No	Replace match predicate (pass `{}` to clear). Refer to match_when predicates reference for the grammar.
`cache_ttl_secs`	int32	No	Update cache TTL (`0` disables)
`cache_scope`	string	No	Update cache scope
`routing_strategy`	string	No	`priority_weighted`, `round_robin`, or `least_loaded`
`strategy_scope`	string	No	`workspace` or `caller`
`failover_max`	string	No	`"all"` (walk every member) or non-negative integer `N`. Omit to leave unchanged.

Permission: gateway.manage

dome_model_pool_set_default

Set the workspace’s default pool. Replaces any prior default.

Parameter	Type	Required	Description
`name`	string	Yes	Pool name

Permission: gateway.manage

dome_model_pool_move

Reposition a pool in its workspace’s match_when evaluation order. Pools with non-empty predicates evaluate in ascending sort_key; the first match wins. Exactly one of before or after is required.

Parameter	Type	Required	Description
`name`	string	Yes	Pool to move
`before`	string	No	Anchor pool name; the moved pool lands immediately before it
`after`	string	No	Anchor pool name; the moved pool lands immediately after it

Permission: gateway.manage

dome_model_pool_remove

Remove an LLM pool. Marked destructive.

Parameter	Type	Required	Description
`name`	string	Yes	Pool name

Permission: gateway.manage

Pool Member CRUD

Members are model connections inside a pool, each with a priority and weight. Failover walks ascending priority; within a priority bucket, requests distribute by weight.

dome_model_pool_member_list

List members of a pool.

Parameter	Type	Required	Description
`pool`	string	Yes	Pool name

Permission: gateway.view

dome_model_pool_member_add

Add a model to a pool.

Parameter	Type	Required	Description
`pool`	string	Yes	Pool name
`model`	string	Yes	Model connection name
`priority`	int32	No	Priority bucket (`0` = primary, ascending fails over)
`weight`	int32	No	Weight within the priority bucket (default: `1`)
`enabled`	bool	No	Whether the member is included in routing (default: `true`)

Permission: gateway.manage

Add a member with failover

{
  "name": "dome_model_pool_member_add",
  "arguments": {
    "pool": "default",
    "model": "claude-sonnet",
    "priority": 0,
    "weight": 1
  }
}

dome_model_pool_member_update

Update a pool member’s priority, weight, or enabled flag.

Parameter	Type	Required	Description
`pool`	string	Yes	Pool name
`model`	string	Yes	Model connection name
`priority`	int32	No	New priority
`weight`	int32	No	New weight
`enabled`	bool	No	New enabled flag

Permission: gateway.manage

dome_model_pool_member_remove

Remove a model from a pool. Marked destructive.

Parameter	Type	Required	Description
`pool`	string	Yes	Pool name
`model`	string	Yes	Model connection name

Permission: gateway.manage

Cost quotas

Cost quotas cap USD spend over a window and gate routing — total caps reject with HTTP 429 when exhausted, while per-model budgets lock a model out of routing so the pool spills to the next member. Refer to Cost quotas for the enforcement model and subjects.

dome_model_quota_list

List LLM cost quotas in the active workspace.

Parameter	Type	Required	Description
—	—	—	No parameters

Permission: gateway.view

dome_model_quota_set

Create a USD spend cap. The subject decides what the cap covers.

Parameter	Type	Required	Description
`subject`	string	Yes	`workspace`, `agent`, `act-as`, `pool`, or `model`
`limit_usd`	number	Yes	Spend cap in US dollars (e.g. `500` or `12.50`)
`window`	string	No	Reset window: `daily` or `monthly` (default `monthly`)
`name`	string	No	Optional human-readable label
`model`	string	Conditional	Model connection name (`subject=model`)
`pool`	string	Conditional	Pool name (`subject=pool`, or to scope a model quota to one pool)
`agent`	string	Conditional	Agent id (`subject=agent`)
`act_as`	string	Conditional	Verified end-user OIDC subject (`subject=act-as`)
`per_caller`	bool	No	`subject=model` only: apply the limit independently per agent and per verified end user
`disabled`	bool	No	Create disabled; no enforcement until enabled

Permission: gateway.manage

Per-model in-pool budget (drives spill-over)

{
  "name": "dome_model_quota_set",
  "arguments": {
    "subject": "model",
    "model": "claude-sonnet",
    "pool": "production",
    "limit_usd": 1000,
    "window": "monthly"
  }
}

dome_model_quota_remove

Remove a cost quota by id. Marked destructive.

Parameter	Type	Required	Description
`quota_id`	string	Yes	Quota id (from `dome_model_quota_list`)

Permission: gateway.manage

Next steps

Add connections and pools from Connect when you want the full workflow:

​Model CRUD

​dome_model_list

​dome_model_get

​dome_model_add

​dome_model_update

​dome_model_remove

​dome_model_oauth_connect

​dome_model_oauth_disconnect

​Pool CRUD

​dome_model_pool_list

​dome_model_pool_get

​dome_model_pool_create

​dome_model_pool_update

​dome_model_pool_set_default

​dome_model_pool_move

​dome_model_pool_remove

​Pool Member CRUD

​dome_model_pool_member_list

​dome_model_pool_member_add

​dome_model_pool_member_update

​dome_model_pool_member_remove

​Cost quotas

​dome_model_quota_list

​dome_model_quota_set

​dome_model_quota_remove

​Next steps

Model CRUD

dome_model_list

dome_model_get

dome_model_add

dome_model_update

dome_model_remove

dome_model_oauth_connect

dome_model_oauth_disconnect

Pool CRUD

dome_model_pool_list

dome_model_pool_get

dome_model_pool_create

dome_model_pool_update

dome_model_pool_set_default

dome_model_pool_move

dome_model_pool_remove

Pool Member CRUD

dome_model_pool_member_list

dome_model_pool_member_add

dome_model_pool_member_update

dome_model_pool_member_remove

Cost quotas

dome_model_quota_list

dome_model_quota_set

dome_model_quota_remove

Next steps