Pools

Overview

A pool contains one or more models. When an agent puts the pool name in a request’s model field, Dome orders the members according to the routing strategy. Dome sends the request to the first eligible member. If that attempt fails, Dome can try the remaining eligible members. The typical workflow is:

Connect the models that can serve the traffic.
Create a pool.
Add the models as members.
Configure the pool’s routing strategy and failover.
Attach the pool to a Gateway.
Allow the Gateway on the agent.
Verify the pool with that agent’s API key.

Members

Each member represents one model connection in the pool. With priority-weighted routing, the priority places the member in a failover tier. The weight determines how often Dome selects the member within that tier. Round-robin and least-loaded routing treat every member equally, so they ignore priority and weight. A model can belong to more than one pool. Removing a member changes only that pool and leaves the model connection available elsewhere.

The management interfaces store an enabled value, but the current gateway routes to members regardless of that value. To stop sending traffic to a member, remove it from the pool.

Routing strategies

The routing strategy chooses the primary model and orders any failover candidates.

Strategy	Primary selection	Member fields it uses
`priority_weighted`	Chooses by weight from the lowest priority tier	Priority and weight
`round_robin`	Rotates through all members in order	Neither
`least_loaded`	Chooses the member with the fewest in-flight requests and breaks ties randomly	Neither

Use priority_weighted when some models should take traffic before others. Use round_robin or least_loaded when members are peers. The strategy_scope setting determines whether round-robin rotation and least-loaded request counts are shared across the workspace or tracked per agent. Priority-weighted routing does not use this setting. You can choose a strategy when you create a pool or change it later.

Failover

The failover limit controls how many additional members Dome may try when the request to the primary member fails.

Value	Runtime behavior
No limit or `all`	Try every eligible member until one succeeds
`0`	Try only the primary
`N`	Try the primary and up to `N` additional members

Dome authorizes the primary before sending the request. If the primary fails, Dome authorizes each remaining member before trying it and skips any member that rules deny. A streaming request can fail over only before Dome sends the first response data to the caller. An error after streaming begins terminates the response. You can set failover when you create a pool or change it later.

Pool resolution

When a request reaches a Gateway, Dome resolves its model value in this order:

A pool with that exact name
A direct model connection with that exact name
A direct connection whose upstream model identifier matches
The first pool in the configured order whose match_when predicate matches
The workspace default pool

An exact pool or model name takes precedence over conditional routing. When the requested name does not match directly, a match_when condition can select a pool based on the request or verified identity (for example prompt_tokens.gt). The full match_when dialect is on the Pools reference. You can set a condition when you create a pool, change it later, and move the pool to control evaluation order.

Response caching

Pool records store a cache TTL and cache scope for reusing responses to identical requests.

The gateway does not read these settings during model dispatch. Setting cache_ttl_secs or cache_scope does not cache responses.

Requirements

Before you begin:

Authenticate to Dome and select a workspace
Connect at least one model
Have a Gateway ready when you want the pool reachable

Permissions

Pool operations require platform permissions. Each operation states its required permission inline.

Default roles	Permission	Grants
All workspace roles	`gateway.view`	List and inspect pools
`admin`, `operator`	`gateway.manage`	Create, update, and remove pools and members

Create pool

Create a pool with a name that clients can put in the request’s model field. A new pool has no members until you add them.

Requires gateway.manage.

CLI
MCP
API
Agent

dome model pool create production \
  --routing-strategy priority_weighted \
  --failover-max all \
  --gateway prod-llms

By default, the CLI uses priority-weighted routing and shares routing state across the workspace. It sets the cache TTL to 0 and allows Dome to try every eligible member after a failure.

Reference: dome model pool create

Tool: dome_model_pool_create

{
  "name": "production",
  "routing_strategy": "priority_weighted",
  "failover_max": "all",
  "gateways": ["prod-llms"]
}

Reference: dome_model_pool_create

POST /dome.mgmt.v1.Management/CreateLLMPool
Content-Type: application/json

{
  "workspace_id": "{{WORKSPACE_ID}}",
  "name": "production",
  "routing_strategy": "priority_weighted",
  "strategy_scope": "workspace"
}

Omit failover_max to let Dome try every eligible member after a failure. The endpoint returns the pool’s UUID. The other management endpoints use this UUID to identify the pool. Attach the pool to a Gateway in a separate operation.

Reference: CreateLLMPool

Create a model pool

Create a model pool named "production" with priority-weighted routing and no failover limit. Attach it to the "prod-llms" Gateway.

Agents cannot call a pool until it has at least one model member and belongs to a Gateway.

Update pool

Update a pool to change its name, description, routing strategy, conditional routing criteria, cache settings, or failover limit. Setting a different default pool and changing the order of conditional routes are separate operations.

Requires gateway.manage.

CLI
MCP
API
Agent

dome model pool update production \
  --description "Production chat routing"

Only the flags you pass change.

Reference: dome model pool update

Tool: dome_model_pool_update

{
  "name": "production",
  "description": "Production chat routing"
}

Only the parameters you send change.

Reference: dome_model_pool_update

POST /dome.mgmt.v1.Management/UpdateLLMPool
Content-Type: application/json

{
  "workspace_id": "{{WORKSPACE_ID}}",
  "pool_id": "{{POOL_ID}}",
  "description": "Production chat routing"
}

Omitted fields keep their stored values. An empty description does not clear the existing description. To clear match_when, set match_when_provided to true. To set the cache TTL to 0, set cache_ttl_secs_provided to true. To let Dome retry every eligible member, set failover_max_clear to true.

Reference: UpdateLLMPool

Update a model pool

Change the description of "production" to "Production chat routing". Keep its members and routing configuration unchanged.

Change routing

You can change how the pool selects its primary member. Refer to Routing strategies for when to use each strategy. When switching away from priority-weighted routing, existing priorities and weights remain stored but no longer affect selection.

CLI
MCP
API
Agent

dome model pool update production \
  --routing-strategy round_robin \
  --strategy-scope caller

Reference: dome model pool update

Tool: dome_model_pool_update

{
  "name": "production",
  "routing_strategy": "round_robin",
  "strategy_scope": "caller"
}

Reference: dome_model_pool_update

POST /dome.mgmt.v1.Management/UpdateLLMPool
Content-Type: application/json

{
  "workspace_id": "{{WORKSPACE_ID}}",
  "pool_id": "{{POOL_ID}}",
  "routing_strategy": "round_robin",
  "strategy_scope": "caller"
}

Reference: UpdateLLMPool

Change pool routing

Change "production" to caller-scoped round-robin routing.

Change request routing

You can replace the conditions that select this pool during conditional routing. Send an empty match_when object to remove those conditions.

CLI
MCP
API
Agent

dome model pool update long-context \
  --match-when '{"prompt_tokens":{"gt":50000}}'

Reference: dome model pool update

Tool: dome_model_pool_update

{
  "name": "long-context",
  "match_when": {
    "prompt_tokens": {
      "gt": 50000
    }
  }
}

Reference: dome_model_pool_update

POST /dome.mgmt.v1.Management/UpdateLLMPool
Content-Type: application/json

{
  "workspace_id": "{{WORKSPACE_ID}}",
  "pool_id": "{{POOL_ID}}",
  "match_when": {
    "prompt_tokens": {
      "gt": 50000
    }
  },
  "match_when_provided": true
}

Reference: UpdateLLMPool

Change conditional routing

Route prompts over 50,000 tokens through the "long-context" pool.

Change failover

You can change how many additional members Dome may try after the primary fails. Refer to Failover for limit values and streaming behavior. After setting a numeric limit, you can use the API to let Dome try every eligible member again.

CLI
MCP
API
Agent

dome model pool update production \
  --failover-max 2

Reference: dome model pool update

Tool: dome_model_pool_update

{
  "name": "production",
  "failover_max": "2"
}

Reference: dome_model_pool_update

POST /dome.mgmt.v1.Management/UpdateLLMPool
Content-Type: application/json

{
  "workspace_id": "{{WORKSPACE_ID}}",
  "pool_id": "{{POOL_ID}}",
  "failover_max": 2
}

To let Dome try every eligible member, send a separate update with failover_max_clear.

POST /dome.mgmt.v1.Management/UpdateLLMPool
Content-Type: application/json

{
  "workspace_id": "{{WORKSPACE_ID}}",
  "pool_id": "{{POOL_ID}}",
  "failover_max_clear": true
}

Reference: UpdateLLMPool

Change pool failover

Allow the "production" pool to retry up to two additional models.

Get pool

Retrieve a pool with its routing configuration and members. Check this information before an update or when you need to determine which models can serve requests to the pool.

Requires gateway.view.

CLI
MCP
API
Agent

dome model pool get production

Reference: dome model pool get

Tool: dome_model_pool_get

{
  "name": "production"
}

Reference: dome_model_pool_get

POST /dome.mgmt.v1.Management/GetLLMPool
Content-Type: application/json

{
  "workspace_id": "{{WORKSPACE_ID}}",
  "pool_id": "{{POOL_ID}}"
}

The API returns the pool and its members.

Reference: GetLLMPool

Get a model pool

Get the "production" pool with its routing configuration and members.

List pools

Retrieve the pools in a workspace with their routing strategies, conditional routing criteria, and default status. Use the list to find a pool to inspect, reorder, update, or remove.

Requires gateway.view.

CLI
MCP
API
Agent

dome model pool list

Reference: dome model pool list

Tool: dome_model_pool_list

{}

Reference: dome_model_pool_list

POST /dome.mgmt.v1.Management/ListLLMPools
Content-Type: application/json

{
  "workspace_id": "{{WORKSPACE_ID}}"
}

Reference: ListLLMPools

List model pools

List the model pools in this workspace with their routing strategy, predicate, and default state.

Set default pool

Set a pool as the workspace default. Dome uses the default only when a request does not match an exact model name, exact pool name, or conditional route. Setting a new default removes the previous default.

Requires gateway.manage.

CLI
MCP
API
Agent

dome model pool set-default production

Reference: dome model pool set-default

Tool: dome_model_pool_set_default

{
  "name": "production"
}

Reference: dome_model_pool_set_default

POST /dome.mgmt.v1.Management/SetDefaultLLMPool
Content-Type: application/json

{
  "workspace_id": "{{WORKSPACE_ID}}",
  "pool_id": "{{POOL_ID}}"
}

Send an empty pool_id to clear the default without setting another pool.

Reference: SetDefaultLLMPool

Set the default model pool

Set "production" as the default model pool for this workspace.

Move pool

Move a conditional pool earlier or later in the order Dome checks match_when predicates. This order affects only conditional routing and does not change routes that use an exact model or pool name.

Requires gateway.manage.

CLI
MCP
Agent

dome model pool move long-context \
  --before production

Pass exactly one of --before or --after.

Reference: dome model pool move

Tool: dome_model_pool_move

{
  "name": "long-context",
  "before": "production"
}

Reference: dome_model_pool_move

Move a conditional pool

Evaluate "long-context" immediately before "production".

Manage members

Pool members are the model connections that Dome can select to serve requests to the pool.

Add member

Add a model to a pool. For a priority-weighted pool, priority defaults to 0 and weight defaults to 1. Round-robin and least-loaded pools reject explicit priority or weight values.

Requires gateway.manage.

CLI
MCP
API
Agent

dome model pool member add production claude-prod \
  --priority 0 \
  --weight 4

Run a separate command for each model because each member can have a different priority and weight.

Reference: dome model pool member add

Tool: dome_model_pool_member_add

{
  "pool": "production",
  "connection": "claude-prod",
  "priority": 0,
  "weight": 4
}

Reference: dome_model_pool_member_add

POST /dome.mgmt.v1.Management/CreateLLMPoolMember
Content-Type: application/json

{
  "workspace_id": "{{WORKSPACE_ID}}",
  "pool_id": "{{POOL_ID}}",
  "llm_model_connection_id": "{{CONNECTION_ID}}",
  "priority": 0,
  "weight": 4,
  "enabled": true
}

Reference: CreateLLMPoolMember

Add a pool member

Add "claude-prod" to "production" at priority zero with weight four.

List members

Retrieve the models in a pool with each member’s priority, weight, and stored enabled value.

Requires gateway.view.

CLI
MCP
API
Agent

dome model pool member list production

Reference: dome model pool member list

Tool: dome_model_pool_member_list

{
  "pool": "production"
}

Reference: dome_model_pool_member_list

POST /dome.mgmt.v1.Management/ListLLMPoolMembers
Content-Type: application/json

{
  "workspace_id": "{{WORKSPACE_ID}}",
  "pool_id": "{{POOL_ID}}"
}

Reference: ListLLMPoolMembers

List pool members

List the members of "production" with their priority and weight.

Update member

Update the priority or weight of a member in a priority-weighted pool. To move a model to another pool, remove it from the current pool and add it to the destination pool.

Requires gateway.manage.

CLI
MCP
API
Agent

dome model pool member update production claude-prod \
  --priority 1 \
  --weight 1

Reference: dome model pool member update

Tool: dome_model_pool_member_update

{
  "pool": "production",
  "connection": "claude-prod",
  "priority": 1,
  "weight": 1
}

Reference: dome_model_pool_member_update

POST /dome.mgmt.v1.Management/UpdateLLMPoolMember
Content-Type: application/json

{
  "workspace_id": "{{WORKSPACE_ID}}",
  "pool_id": "{{POOL_ID}}",
  "member_id": "{{MEMBER_ID}}",
  "priority": 1,
  "weight": 1
}

Reference: UpdateLLMPoolMember

Update a pool member

Change "claude-prod" in "production" to priority one and weight one.

Remove member

Remove a model from a pool to stop the pool from sending requests to that model. Removing the member does not delete the model connection, so other pools and direct Gateway routes can still use it.

Requires gateway.manage.

CLI
MCP
API
Agent

dome model pool member rm production claude-prod

Reference: dome model pool member rm

Tool: dome_model_pool_member_remove

{
  "pool": "production",
  "connection": "claude-prod"
}

Reference: dome_model_pool_member_remove

POST /dome.mgmt.v1.Management/DeleteLLMPoolMember
Content-Type: application/json

{
  "workspace_id": "{{WORKSPACE_ID}}",
  "pool_id": "{{POOL_ID}}",
  "member_id": "{{MEMBER_ID}}"
}

Reference: DeleteLLMPoolMember

Remove a pool member

Remove "claude-prod" from "production" without deleting the model.

Remove pool

Remove a pool to delete its routing configuration and memberships. Removing the pool does not delete its model connections.

Requires gateway.manage.

Removing a pool also removes it from every Gateway. Requests that use the deleted pool name fail unless a conditional route or the workspace default selects another pool.

CLI
MCP
API
Agent

dome model pool rm production

Reference: dome model pool rm

Tool: dome_model_pool_remove

{
  "name": "production"
}

Reference: dome_model_pool_remove

POST /dome.mgmt.v1.Management/DeleteLLMPool
Content-Type: application/json

{
  "workspace_id": "{{WORKSPACE_ID}}",
  "pool_id": "{{POOL_ID}}"
}

Reference: DeleteLLMPool

Remove a model pool

Remove the "production" pool after confirming no clients still use its name.

Attach to a Gateway

Attach a pool to a Gateway so agents can request the pool by name at that gateway’s model endpoints. The attachment makes the pool reachable but does not authorize an agent to call it. The agent must allow the Gateway, and rules must permit both the pool and selected model.

Requires gateway.manage.

CLI
MCP
Agent

dome model pool gateways add production prod-llms

Reference: dome model pool gateways add

Tool: dome_gateway_model_pool_add

{
  "gateway": "prod-llms",
  "pool": "production"
}

Reference: dome_gateway_model_pool_add

Publish a model pool

Add the "production" model pool to the "prod-llms" Gateway.

Verify pool

Call the pool through the Gateway with an agent API key. A successful response confirms the pool is reachable, rules permit the route, and at least one member can serve the request.

OpenAI-compatible client

from openai import OpenAI

client = OpenAI(
    api_key="{{DOME_AGENT_KEY}}",
    base_url="https://{{GATEWAY_HOST}}/gateways/{{GATEWAY_ID}}/v1",
)

response = client.chat.completions.create(
    model="production",
    messages=[{"role": "user", "content": "Return OK."}],
)

If Dome cannot find the pool for the request, check the pool’s Gateway membership and model members. If Dome denies the request, check the agent’s allowed resources and rules. Audit events record which model served the request and whether Dome attempted failover.

Next steps

Pools concept for how routing and failover work and Pools reference for strategies and match_when
Connect more models to add providers or failover capacity
Manage membership and grants in Gateways
Configure model and pool spend limits in Cost Quotas

Get started

Core concepts

Connect

Govern

Operate

Manage

Develop

Reference

Overview

Members

Routing strategies

Failover

Pool resolution

Response caching

Requirements

Permissions

Create pool

Update pool

Change routing

Change request routing

Change failover

Get pool

List pools

Set default pool

Move pool

Manage members

Add member

List members

Update member

Remove member

Remove pool

Attach to a Gateway

Verify pool

Next steps

​Overview

​Members

​Routing strategies

​Failover

​Pool resolution

​Response caching

​Requirements

​Permissions

​Create pool

​Update pool

​Change routing

​Change request routing

​Change failover

​Get pool

​List pools

​Set default pool

​Move pool

​Manage members

​Add member

​List members

​Update member

​Remove member

​Remove pool

​Attach to a Gateway

​Verify pool

​Next steps

Overview

Members

Routing strategies

Failover

Pool resolution

Response caching

Requirements

Permissions

Create pool

Update pool

Change routing

Change request routing

Change failover

Get pool

List pools

Set default pool

Move pool

Manage members

Add member

List members

Update member

Remove member

Remove pool

Attach to a Gateway

Verify pool

Next steps