Pools

A pool groups multiple model connections behind one name. Agents request the pool by name. Dome routes, load-balances, and fails over across members.

High-level overview

A pool contains one or more model connections as members. When an agent puts the pool name in the request's model field, Dome orders eligible members by the routing strategy, sends the request to the primary, and may try further members on failure up to the failover limit.

For an agent to use a pool:

1. Connect the models that can serve the traffic.
2. Create the pool and add members.
3. Configure routing and failover.
4. Attach the pool to a Gateway.
5. Allow that Gateway on the agent.
6. Verify with the agent's key.

Routing through a pool looks like this:

1. An operator creates production with priority-weighted routing, adds claude-prod at priority 0 and gpt-backup at priority 1, attaches the pool to prod-llms, and grants an agent access.
2. The agent calls with model: "production".
3. Dome resolves the pool, authorizes the primary, and dispatches.
4. On a transient provider failure, Dome may authorize and try the backup.
5. Audit records which member served the call.

A pool is the stable name clients and Rules use. Members and strategy stay editable underneath.

Members

Each member is one model connection. With priority-weighted routing, priority places the member in a failover tier and weight shares traffic inside the tier. Round-robin, least-loaded, and most-quota-remaining ignore priority and weight. A model can belong to more than one pool. Removing a member leaves the connection available elsewhere.

Routing strategies

The strategy chooses the primary and orders failover candidates: priority_weighted, round_robin, least_loaded, or most_quota_remaining. Strategy scope (workspace or caller) controls whether stateful strategies share rotation or inflight state. For most_quota_remaining, scope only affects the tiebreak — primary order follows remaining cost-quota headroom. Catalog is on the Pools reference.

Failover

Failover limits how many additional members Dome may try after the primary fails. Dome authorizes the primary first, then re-authorizes each candidate before trying it. Streaming can fail over only before the first response byte reaches the caller. Limits and behavior are on the Pools reference.

Pool resolution

When a request reaches a Gateway, Dome resolves the model value in a fixed order: exact pool name, exact connection name, upstream model id match, first matching match_when pool, then the workspace default pool. Exact names beat conditional routes. The match_when dialect is on the Pools reference. Routing does not replace authorization. Rules still evaluate the pool and selected model.

Response caching

Pool records store cache TTL and scope fields intended for identical-request reuse.

Next steps

With that model-pool routing model in mind, continue with:

• Resources concept for how pools sit among resource types
• Models concept for the connections pools select
• LLM gateway concept for dispatch and failover on the wire
• Quotas concept for spend caps and pool spillover
• Pools reference for strategies, failover, and match_when
• Pools to create pools, manage members, and attach Gateways