Pools

A pool groups multiple model connections behind one name. Agents request the pool by name. Dome routes, load-balances, and fails over across members.

Overview

A pool contains one or more models. When an agent puts the pool name in a request's model field, Dome orders the members according to the routing strategy. Dome sends the request to the first eligible member. If that attempt fails, Dome can try the remaining eligible members.

The typical workflow is:

Connect the models that can serve the traffic.
Create a pool.
Add the models as members.
Configure the pool's routing strategy and failover.
Attach the pool to a Gateway.
Allow the Gateway on the agent.
Verify the pool with that agent's API key.

Members

Each member represents one model connection in the pool. With priority-weighted routing, the priority places the member in a failover tier. The weight determines how often Dome selects the member within that tier. Round-robin, least-loaded, and most-quota-remaining routing treat every member equally, so they ignore priority and weight.

A model can belong to more than one pool. Removing a member changes only that pool and leaves the model connection available elsewhere.

The management interfaces store an enabled value, but the current gateway routes to members regardless of that value. To stop sending traffic to a member, remove it from the pool.

Routing strategies

The routing strategy chooses the primary model and orders any failover candidates.

Strategy	Primary selection	Member fields it uses
`priority_weighted`	Chooses by weight from the lowest priority tier	Priority and weight
`round_robin`	Rotates through all members in order	Neither
`least_loaded`	Chooses the member with the fewest in-flight requests and breaks ties randomly	Neither
`most_quota_remaining`	Chooses the member with the most remaining per-model cost-quota headroom; unmetered members serve last	Neither

Use priority_weighted when some models should take traffic before others. Use round_robin or least_loaded when members are peers. Use most_quota_remaining when members carry different per-model cost caps and you want to drain the highest-headroom member first. Exhausted members are locked out by the quota barrier before this strategy orders the survivors. With no per-model quotas, every member ties and the strategy degrades to round-robin — meter every member for the ordering to have effect.

The strategy_scope setting determines whether round-robin rotation, least-loaded request counts, and most_quota_remaining tiebreaks are shared across the workspace or tracked per agent. Priority-weighted routing does not use this setting. For most_quota_remaining, scope only affects the tiebreak — not the primary ordering.

You can choose a strategy when you create a pool or change it later.

Failover

The failover limit controls how many additional members Dome may try when the request to the primary member fails.

Value	Runtime behavior
No limit or `all`	Try every eligible member until one succeeds
`0`	Try only the primary
`N`	Try the primary and up to `N` additional members

Dome authorizes the primary before sending the request. If the primary fails, Dome authorizes each remaining member before trying it and skips any member that rules deny. A streaming request can fail over only before Dome sends the first response data to the caller. An error after streaming begins terminates the response.

You can set failover when you create a pool or change it later.

Pool resolution

When a request reaches a Gateway, Dome resolves its model value in this order:

A pool with that exact name
A direct model connection with that exact name
A direct connection whose upstream model identifier matches
The first pool in the configured order whose match_when predicate matches
The workspace default pool

An exact pool or model name takes precedence over conditional routing. When the requested name does not match directly, a match_when condition can select a pool based on the request or verified identity (for example prompt_tokens.gt). The full match_when dialect is on the Pools reference. You can set a condition when you create a pool, change it later, and move the pool to control evaluation order.

Response caching

Pool records store a cache TTL and cache scope for reusing responses to identical requests.

The gateway does not read these settings during model dispatch. Setting cache_ttl_secs or cache_scope does not cache responses.

Requirements

Before you begin:

Authenticate to Dome and select a workspace
Connect at least one model
Have a Gateway ready when you want the pool reachable

Permissions

Pool operations require platform permissions. Each operation states its required permission inline.

Default roles	Permission	Grants
All workspace roles	`gateways.view`	List and inspect pools
`admin`, `operator`	`gateways.manage`	Create, update, and remove pools and members

Create pool

Create a pool with a name that clients can put in the request's model field. A new pool has no members until you add them.

Requires gateways.manage.

dome models pool create production \
  --routing-strategy priority_weighted \
  --failover-max all \
  --gateway prod-llms

By default, the CLI uses priority-weighted routing and shares routing state across the workspace. It sets the cache TTL to 0 and allows Dome to try every eligible member after a failure.

Reference: dome models pool create

Tool: dome_models_pool_create

{
  "name": "production",
  "routing_strategy": "priority_weighted",
  "failover_max": "all",
  "gateways": ["prod-llms"]
}

Reference: dome_models_pool_create

POST /v1/models/pools
Content-Type: application/json

{
  "workspace_id": "{{WORKSPACE_ID}}",
  "name": "production",
  "routing_strategy": "priority_weighted",
  "strategy_scope": "workspace"
}

Omit failover_max to let Dome try every eligible member after a failure. The endpoint returns the pool's UUID. The other management endpoints use this UUID to identify the pool. Attach the pool to a Gateway in a separate operation.

Reference: CreateLLMPool

Create a model pool

Create a model pool named "production" with priority-weighted routing and no failover limit. Attach it to the "prod-llms" Gateway.

Agents cannot call a pool until it has at least one model member and belongs to a Gateway.

Update pool

Update a pool to change its name, description, routing strategy, conditional routing criteria, cache settings, or failover limit. Setting a different default pool and changing the order of conditional routes are separate operations.

Requires gateways.manage.

dome models pool update production \
  --description "Production chat routing"

Only the flags you pass change.

Reference: dome models pool update

Tool: dome_models_pool_update

{
  "name": "production",
  "description": "Production chat routing"
}

Only the parameters you send change.

Reference: dome_models_pool_update

PATCH /v1/models/pools/{{POOL_ID}}
Content-Type: application/json

{
  "workspace_id": "{{WORKSPACE_ID}}",
  "description": "Production chat routing"
}

Omitted fields keep their stored values. An empty description does not clear the existing description. To clear match_when, set match_when_provided to true. To set the cache TTL to 0, set cache_ttl_secs_provided to true. To let Dome retry every eligible member, set failover_max_clear to true.

Reference: UpdateLLMPool

Update a model pool

Change the description of "production" to "Production chat routing". Keep its members and routing configuration unchanged.

Change routing

You can change how the pool selects its primary member. Refer to Routing strategies for when to use each strategy. When switching away from priority-weighted routing, existing priorities and weights remain stored but no longer affect selection.

dome models pool update production \
  --routing-strategy most_quota_remaining \
  --strategy-scope workspace

Reference: dome models pool update

Tool: dome_models_pool_update

{
  "name": "production",
  "routing_strategy": "most_quota_remaining",
  "strategy_scope": "workspace"
}

Reference: dome_models_pool_update

PATCH /v1/models/pools/{{POOL_ID}}
Content-Type: application/json

{
  "workspace_id": "{{WORKSPACE_ID}}",
  "routing_strategy": "most_quota_remaining",
  "strategy_scope": "workspace"
}

Reference: UpdateLLMPool

Change pool routing

Change "production" to caller-scoped round-robin routing.

Change request routing

You can replace the conditions that select this pool during conditional routing. Send an empty match_when object to remove those conditions.

dome models pool update long-context \
  --match-when '{"prompt_tokens":{"gt":50000}}'

Reference: dome models pool update

Tool: dome_models_pool_update

{
  "name": "long-context",
  "match_when": {
    "prompt_tokens": {
      "gt": 50000
    }
  }
}

Reference: dome_models_pool_update

PATCH /v1/models/pools/{{POOL_ID}}
Content-Type: application/json

{
  "workspace_id": "{{WORKSPACE_ID}}",
  "match_when": {
    "prompt_tokens": {
      "gt": 50000
    }
  },
  "match_when_provided": true
}

Reference: UpdateLLMPool

Change conditional routing

Route prompts over 50,000 tokens through the "long-context" pool.

Change failover

You can change how many additional members Dome may try after the primary fails. Refer to Failover for limit values and streaming behavior. After setting a numeric limit, you can use the API to let Dome try every eligible member again.

dome models pool update production \
  --failover-max 2

Reference: dome models pool update

Tool: dome_models_pool_update

{
  "name": "production",
  "failover_max": "2"
}

Reference: dome_models_pool_update

PATCH /v1/models/pools/{{POOL_ID}}
Content-Type: application/json

{
  "workspace_id": "{{WORKSPACE_ID}}",
  "failover_max": 2
}

To let Dome try every eligible member, send a separate update with failover_max_clear.

PATCH /v1/models/pools/{{POOL_ID}}
Content-Type: application/json

{
  "workspace_id": "{{WORKSPACE_ID}}",
  "failover_max_clear": true
}

Reference: UpdateLLMPool

Change pool failover

Allow the "production" pool to retry up to two additional models.

Get pool

Retrieve a pool with its routing configuration and members. Check this information before an update or when you need to determine which models can serve requests to the pool.

Requires gateways.view.

dome models pool get production

Reference: dome models pool get

Tool: dome_models_pool_get

{
  "name": "production"
}

Reference: dome_models_pool_get

GET /v1/models/pools/{{POOL_ID}}?workspace_id={{WORKSPACE_ID}}

The API returns the pool and its members.

Reference: GetLLMPool

Get a model pool

Get the "production" pool with its routing configuration and members.

List pools

Retrieve the pools in a workspace with their routing strategies, conditional routing criteria, and default status. Use the list to find a pool to inspect, reorder, update, or remove.

Requires gateways.view.

dome models pool list

Reference: dome models pool list

Tool: dome_models_pool_list

{}

Reference: dome_models_pool_list

GET /v1/models/pools?workspace_id={{WORKSPACE_ID}}

Reference: ListLLMPools

List model pools

List the model pools in this workspace with their routing strategy, predicate, and default state.

Set default pool

Set a pool as the workspace default. Dome uses the default only when a request does not match an exact model name, exact pool name, or conditional route. Setting a new default removes the previous default.

Requires gateways.manage.

dome models pool set-default production

Reference: dome models pool set-default

Tool: dome_models_pool_set_default

{
  "name": "production"
}

Reference: dome_models_pool_set_default

POST /v1/models/pools/{{POOL_ID}}/default
Content-Type: application/json

{
  "workspace_id": "{{WORKSPACE_ID}}"
}

Send an empty pool_id to clear the default without setting another pool.

Reference: SetDefaultLLMPool

Set the default model pool

Set "production" as the default model pool for this workspace.

Move pool

Move a conditional pool earlier or later in the order Dome checks match_when predicates. This order affects only conditional routing and does not change routes that use an exact model or pool name.

Requires gateways.manage.

dome models pool move long-context \
  --before production

Pass exactly one of --before or --after.

Reference: dome models pool move

Tool: dome_models_pool_move

{
  "name": "long-context",
  "before": "production"
}

Reference: dome_models_pool_move

Move a conditional pool

Evaluate "long-context" immediately before "production".

Manage members

Pool members are the model connections that Dome can select to serve requests to the pool.

Add member

Add a model to a pool. For a priority-weighted pool, priority defaults to 0 and weight defaults to 1. Round-robin and least-loaded pools reject explicit priority or weight values.

Requires gateways.manage.

dome models pool member add production claude-prod \
  --priority 0 \
  --weight 4

Run a separate command for each model because each member can have a different priority and weight.

Reference: dome models pool member add

Tool: dome_models_pool_member_add

{
  "pool": "production",
  "connection": "claude-prod",
  "priority": 0,
  "weight": 4
}

Reference: dome_models_pool_member_add

POST /v1/models/pools/{{POOL_ID}}/members
Content-Type: application/json

{
  "workspace_id": "{{WORKSPACE_ID}}",
  "llm_model_connection_id": "{{CONNECTION_ID}}",
  "priority": 0,
  "weight": 4,
  "enabled": true
}

Reference: CreateLLMPoolMember

Add a pool member

Add "claude-prod" to "production" at priority zero with weight four.

List members

Retrieve the models in a pool with each member's priority, weight, and stored enabled value.

Requires gateways.view.

dome models pool member list production

Reference: dome models pool member list

Tool: dome_models_pool_member_list

{
  "pool": "production"
}

Reference: dome_models_pool_member_list

GET /v1/models/pools/{{POOL_ID}}/members?workspace_id={{WORKSPACE_ID}}

Reference: ListLLMPoolMembers

List pool members

List the members of "production" with their priority and weight.

Update member

Update the priority or weight of a member in a priority-weighted pool. To move a model to another pool, remove it from the current pool and add it to the destination pool.

Requires gateways.manage.

dome models pool member update production claude-prod \
  --priority 1 \
  --weight 1

Reference: dome models pool member update

Tool: dome_models_pool_member_update

{
  "pool": "production",
  "connection": "claude-prod",
  "priority": 1,
  "weight": 1
}

Reference: dome_models_pool_member_update

PATCH /v1/models/pools/{{POOL_ID}}/members/{{MEMBER_ID}}
Content-Type: application/json

{
  "workspace_id": "{{WORKSPACE_ID}}",
  "priority": 1,
  "weight": 1
}

Reference: UpdateLLMPoolMember

Update a pool member

Change "claude-prod" in "production" to priority one and weight one.

Remove member

Remove a model from a pool to stop the pool from sending requests to that model. Removing the member does not delete the model connection, so other pools and direct Gateway routes can still use it.

Requires gateways.manage.

dome models pool member rm production claude-prod

Reference: dome models pool member rm

Tool: dome_models_pool_member_remove

{
  "pool": "production",
  "connection": "claude-prod"
}

Reference: dome_models_pool_member_remove

DELETE /v1/models/pools/{{POOL_ID}}/members/{{MEMBER_ID}}?workspace_id={{WORKSPACE_ID}}

Reference: DeleteLLMPoolMember

Remove a pool member

Remove "claude-prod" from "production" without deleting the model.

Remove pool

Remove a pool to delete its routing configuration and memberships. Removing the pool does not delete its model connections.

Requires gateways.manage.

Removing a pool also removes it from every Gateway. Requests that use the deleted pool name fail unless a conditional route or the workspace default selects another pool.

dome models pool rm production

Reference: dome models pool rm

Tool: dome_models_pool_remove

{
  "name": "production"
}

Reference: dome_models_pool_remove

DELETE /v1/models/pools/{{POOL_ID}}?workspace_id={{WORKSPACE_ID}}

Reference: DeleteLLMPool

Remove a model pool

Remove the "production" pool after confirming no clients still use its name.

Pool activity metrics

Read pool operational metrics from GET /api/v1/metrics/pools/{pool_id} (workspace-scoped; same time-range params as other metrics routes). The response includes failover_served_calls, request_failures, a members breakdown, and time-series buckets for the window.

Use failover_served_calls to see how often traffic landed on a non-primary member after a failure. LLM audit events also carry pool_id so you can filter spend and denials by pool. Workspace and per-agent totals are on Token usage and cost.

Attach to a Gateway

Attach a pool to a Gateway so agents can request the pool by name at that gateway's model endpoints. The attachment makes the pool reachable but does not authorize an agent to call it. The agent must allow the Gateway, and rules must permit both the pool and selected model.

Requires gateways.manage.

dome models pool gateways add production prod-llms

Reference: dome models pool gateways add

Tool: dome_gateways_model_pool_add

{
  "gateway": "prod-llms",
  "pool": "production"
}

Reference: dome_gateways_model_pool_add

Publish a model pool

Add the "production" model pool to the "prod-llms" Gateway.

Verify pool

Call the pool through the Gateway with an agent API key. A successful response confirms the pool is reachable, rules permit the route, and at least one member can serve the request.

OpenAI-compatible client

from openai import OpenAI

client = OpenAI(
    api_key="{{DOME_AGENT_KEY}}",
    base_url="https://{{GATEWAY_HOST}}/gateways/{{GATEWAY_ID}}/v1",
)

response = client.chat.completions.create(
    model="production",
    messages=[{"role": "user", "content": "Return OK."}],
)

If Dome cannot find the pool for the request, check the pool's Gateway membership and model members. If Dome denies the request, check the agent's allowed resources and rules. Audit events record which model served the request and whether Dome attempted failover.

Next steps

Pools concept for how routing and failover work and Pools reference for strategies and match_when
Connect more models to add providers or failover capacity
Manage membership and grants in Gateways
Configure model and pool spend limits in Cost Quotas

Pools

On this page