> ## Documentation Index > Fetch the complete documentation index at: https://docs.domesystems.ai/llms.txt > Use this file to discover all available pages before exploring further. # LLM Models > Configure LLM model connections and routing pools through the gateway Model tools register LLM upstreams the gateway proxies — Anthropic, OpenAI, Azure OpenAI, Google, Bedrock, hosted OpenAI-compatible vendors (Groq, Mistral, Together, Fireworks, DeepSeek, xAI, Perplexity, Cohere, OpenRouter, Cerebras, NVIDIA, DeepInfra, SambaNova, AI21, Databricks), and self-hosted endpoints — and group them into routing pools. Pools give agents one logical name that fans out across providers with priority, weighting, and failover. Models share workspace scope, gateway placement, and credential handling with [Tools](/reference/mcp/gateway). Use models for inference endpoints; use tools for MCP servers. ## Model CRUD ### dome\_model\_list List LLM model connections in the active workspace. | Parameter | Type | Required | Description | | --------- | ---- | -------- | ------------- | | — | — | — | No parameters | **Permission:** `gateway.view` ### dome\_model\_get Get an LLM model connection by name. | Parameter | Type | Required | Description | | --------- | ------ | -------- | ----------- | | `name` | string | Yes | Model name | **Permission:** `gateway.view` ### dome\_model\_add Add an LLM model connection. | Parameter | Type | Required | Description | | ----------------- | ------ | -------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | `name` | string | Yes | Connection name (unique within workspace) | | `provider` | string | Yes | `openai`, `anthropic`, `google`, `bedrock`, `azure_openai`, `mistral`, `groq`, `together`, `fireworks`, `deepseek`, `xai`, `perplexity`, `cohere`, `openrouter`, `cerebras`, `nvidia`, `deepinfra`, `sambanova`, `ai21`, `databricks`, `openai_compatible`, `custom`. Refer to [Supported providers](/reference/resources/models#providers) reference. | | `provider_config` | object | No | Provider-specific config. Common keys: `model`, `endpoint` (omit `endpoint` for hosted providers to use the registry's default base URL) | | `auth_method` | string | No | `none`, `api-key`, `oauth` (default: `api-key`) | | `credential_type` | string | No | `none`, `shared`, `per-user` (default: `shared`) | | `api_key` | string | No | API key for `api-key`+`shared`. Stored under the provider's managed auth header (`Authorization` for OpenAI/Bedrock, `x-api-key` for Anthropic, `api-key` for Azure, `x-goog-api-key` for Google) | | `secret_values` | object | No | Advanced: per-connection credential bundle for custom secret-source headers | | `attributes` | object | No | Cedar attributes used by authorization rules | | `gateways` | array | No | [Gateway](/connect/gateways) names to attach the model to directly at create time. Without it the model is unreachable until attached (`dome_gateway_model_add`). | **Permission:** `gateway.manage` ```json title="Add an Anthropic model" theme={"system"} { "name": "dome_model_add", "arguments": { "name": "claude-sonnet", "provider": "anthropic", "provider_config": { "model": "claude-3-5-sonnet-20241022" }, "api_key": "sk-ant-..." } } ``` ### dome\_model\_update Update an LLM model connection. Only provided fields change. | Parameter | Type | Required | Description | | ----------------- | ------ | -------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `name` | string | Yes | Model name | | `new_name` | string | No | Rename the connection | | `provider_config` | object | No | Replace `provider_config` | | `auth_method` | string | No | New auth method | | `credential_type` | string | No | New credential scope | | `api_key` | string | No | Rotate the shared API key. Stored under the provider's managed auth header (`Authorization` for OpenAI/Bedrock, `x-api-key` for Anthropic, `api-key` for Azure, `x-goog-api-key` for Google) | | `attributes` | object | No | Replace Cedar attributes (pass `{}` to clear) | **Permission:** `gateway.manage` Provider is immutable after create — to switch providers, add a new connection. ### dome\_model\_remove Remove an LLM model connection. Marked destructive. | Parameter | Type | Required | Description | | --------- | ------ | -------- | ----------- | | `name` | string | Yes | Model name | **Permission:** `gateway.manage` ### dome\_model\_oauth\_connect Begin the shared-OAuth admin consent flow for a model configured with `auth_method=oauth`, `credential_type=shared` (Anthropic OAuth, Azure AAD service principal, Google Workload Identity for Vertex). Returns a one-shot URL valid for \~10 minutes. | Parameter | Type | Required | Description | | --------- | ------ | -------- | ----------- | | `name` | string | Yes | Model name | **Permission:** `gateway.manage` ### dome\_model\_oauth\_disconnect Disconnect (revoke) shared-OAuth credentials for a model. Idempotent. Marked destructive. | Parameter | Type | Required | Description | | --------- | ------ | -------- | ----------- | | `name` | string | Yes | Model name | **Permission:** `gateway.manage` ## Pool CRUD Pools are logical names agents call. The gateway picks a member at request time using the pool's routing strategy and falls back across priority buckets when a member fails. ### dome\_model\_pool\_list List LLM pools in the active workspace. **Permission:** `gateway.view` ### dome\_model\_pool\_get Get a pool with its members. | Parameter | Type | Required | Description | | --------- | ------ | -------- | ----------- | | `name` | string | Yes | Pool name | **Permission:** `gateway.view` ### dome\_model\_pool\_create Create an LLM pool. | Parameter | Type | Required | Description | | ------------------ | ------ | -------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | `name` | string | Yes | Pool name (unique within workspace) | | `description` | string | No | Free-form description | | `match_when` | object | No | Predicate selecting this pool by request shape (e.g. `{"prompt_tokens":{"gt":100000}}`). Refer to [match\_when predicates](/reference/resources/model-pools#pool-resolution) reference for the full grammar — operators (`gt`/`lt`/`eq`, `in`, `prefix`, `suffix`, `regex`) and composition (`any`, `all`, `not`). | | `is_default` | bool | No | Mark as the workspace default pool | | `cache_ttl_secs` | int32 | No | Exact-match cache TTL (`0` disables cache) | | `cache_scope` | string | No | `workspace` or `caller` (default: `workspace`) | | `routing_strategy` | string | No | `priority_weighted`, `round_robin`, `least_loaded` (default: `priority_weighted`) | | `strategy_scope` | string | No | Scope of stateful routing counters: `workspace` (default) or `caller`. Ignored by `priority_weighted`. | | `failover_max` | string | No | Failover ceiling. `"all"` walks every member; integer `N` gives primary + N retries (`0` disables failover). Defaults to `"all"`. | | `gateways` | array | No | [Gateway](/connect/gateways) names to attach the pool to at create time. Without it the pool is unreachable until attached (`dome_gateway_model_pool_add`). | **Permission:** `gateway.manage` ### dome\_model\_pool\_update Update a pool. Only provided fields change. | Parameter | Type | Required | Description | | ------------------ | ------ | -------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------ | | `name` | string | Yes | Pool name | | `new_name` | string | No | Rename | | `description` | string | No | Update description | | `match_when` | object | No | Replace match predicate (pass `{}` to clear). Refer to [match\_when predicates](/reference/resources/model-pools#pool-resolution) reference for the grammar. | | `cache_ttl_secs` | int32 | No | Update cache TTL (`0` disables) | | `cache_scope` | string | No | Update cache scope | | `routing_strategy` | string | No | `priority_weighted`, `round_robin`, or `least_loaded` | | `strategy_scope` | string | No | `workspace` or `caller` | | `failover_max` | string | No | `"all"` (walk every member) or non-negative integer `N`. Omit to leave unchanged. | **Permission:** `gateway.manage` ### dome\_model\_pool\_set\_default Set the workspace's default pool. Replaces any prior default. | Parameter | Type | Required | Description | | --------- | ------ | -------- | ----------- | | `name` | string | Yes | Pool name | **Permission:** `gateway.manage` ### dome\_model\_pool\_move Reposition a pool in its workspace's `match_when` evaluation order. Pools with non-empty predicates evaluate in ascending `sort_key`; the first match wins. Exactly one of `before` or `after` is required. | Parameter | Type | Required | Description | | --------- | ------ | -------- | ------------------------------------------------------------ | | `name` | string | Yes | Pool to move | | `before` | string | No | Anchor pool name; the moved pool lands immediately before it | | `after` | string | No | Anchor pool name; the moved pool lands immediately after it | **Permission:** `gateway.manage` ### dome\_model\_pool\_remove Remove an LLM pool. Marked destructive. | Parameter | Type | Required | Description | | --------- | ------ | -------- | ----------- | | `name` | string | Yes | Pool name | **Permission:** `gateway.manage` ## Pool Member CRUD Members are model connections inside a pool, each with a priority and weight. Failover walks ascending priority; within a priority bucket, requests distribute by weight. ### dome\_model\_pool\_member\_list List members of a pool. | Parameter | Type | Required | Description | | --------- | ------ | -------- | ----------- | | `pool` | string | Yes | Pool name | **Permission:** `gateway.view` ### dome\_model\_pool\_member\_add Add a model to a pool. | Parameter | Type | Required | Description | | ---------- | ------ | -------- | ----------------------------------------------------------- | | `pool` | string | Yes | Pool name | | `model` | string | Yes | Model connection name | | `priority` | int32 | No | Priority bucket (`0` = primary, ascending fails over) | | `weight` | int32 | No | Weight within the priority bucket (default: `1`) | | `enabled` | bool | No | Whether the member is included in routing (default: `true`) | **Permission:** `gateway.manage` ```json title="Add a member with failover" theme={"system"} { "name": "dome_model_pool_member_add", "arguments": { "pool": "default", "model": "claude-sonnet", "priority": 0, "weight": 1 } } ``` ### dome\_model\_pool\_member\_update Update a pool member's priority, weight, or enabled flag. | Parameter | Type | Required | Description | | ---------- | ------ | -------- | --------------------- | | `pool` | string | Yes | Pool name | | `model` | string | Yes | Model connection name | | `priority` | int32 | No | New priority | | `weight` | int32 | No | New weight | | `enabled` | bool | No | New enabled flag | **Permission:** `gateway.manage` ### dome\_model\_pool\_member\_remove Remove a model from a pool. Marked destructive. | Parameter | Type | Required | Description | | --------- | ------ | -------- | --------------------- | | `pool` | string | Yes | Pool name | | `model` | string | Yes | Model connection name | **Permission:** `gateway.manage` ## Cost quotas Cost quotas cap USD spend over a window and gate routing — total caps reject with HTTP 429 when exhausted, while per-model budgets lock a model out of routing so the pool spills to the next member. Refer to [Cost quotas](/govern/quotas#set-a-quota) for the enforcement model and subjects. ### dome\_model\_quota\_list List LLM cost quotas in the active workspace. | Parameter | Type | Required | Description | | --------- | ---- | -------- | ------------- | | — | — | — | No parameters | **Permission:** `gateway.view` ### dome\_model\_quota\_set Create a USD spend cap. The `subject` decides what the cap covers. | Parameter | Type | Required | Description | | ------------ | ------ | ----------- | --------------------------------------------------------------------------------------- | | `subject` | string | Yes | `workspace`, `agent`, `act-as`, `pool`, or `model` | | `limit_usd` | number | Yes | Spend cap in US dollars (e.g. `500` or `12.50`) | | `window` | string | No | Reset window: `daily` or `monthly` (default `monthly`) | | `name` | string | No | Optional human-readable label | | `model` | string | Conditional | Model connection name (`subject=model`) | | `pool` | string | Conditional | Pool name (`subject=pool`, or to scope a model quota to one pool) | | `agent` | string | Conditional | Agent id (`subject=agent`) | | `act_as` | string | Conditional | Verified end-user OIDC subject (`subject=act-as`) | | `per_caller` | bool | No | `subject=model` only: apply the limit independently per agent and per verified end user | | `disabled` | bool | No | Create disabled; no enforcement until enabled | **Permission:** `gateway.manage` ```json title="Per-model in-pool budget (drives spill-over)" theme={"system"} { "name": "dome_model_quota_set", "arguments": { "subject": "model", "model": "claude-sonnet", "pool": "production", "limit_usd": 1000, "window": "monthly" } } ``` ### dome\_model\_quota\_remove Remove a cost quota by id. Marked destructive. | Parameter | Type | Required | Description | | ---------- | ------ | -------- | --------------------------------------- | | `quota_id` | string | Yes | Quota id (from `dome_model_quota_list`) | **Permission:** `gateway.manage` ## Next steps Add connections and pools from Connect when you want the full workflow: * [Models](/connect/resources/models) * [Pools](/connect/resources/models/pools) * [Set Usage Limits](/govern/quotas)