High-level overview
A pool contains one or more model connections as members. When an agent puts the pool name in the request’s model field, Dome orders eligible members by the routing strategy, sends the request to the primary, and may try further members on failure up to the failover limit.
For an agent to use a pool:
- Connect the models that can serve the traffic.
- Create the pool and add members.
- Configure routing and failover.
- Attach the pool to a Gateway.
- Allow that Gateway on the agent.
- Verify with the agent’s key.
Routing through a pool looks like this:
- An operator creates
production with priority-weighted routing, adds claude-prod at priority 0 and gpt-backup at priority 1, attaches the pool to prod-llms, and grants an agent access.
- The agent calls with
model: "production".
- Dome resolves the pool, authorizes the primary, and dispatches.
- On a transient provider failure, Dome may authorize and try the backup.
- Audit records which member served the call.
A pool is the stable name clients and Rules use. Members and strategy stay editable underneath.
Members
Each member is one model connection. With priority-weighted routing, priority places the member in a failover tier and weight shares traffic inside the tier. Round-robin and least-loaded ignore priority and weight. A model can belong to more than one pool. Removing a member leaves the connection available elsewhere.
The management interfaces store an enabled value, but the current gateway routes to members regardless of that value. To stop sending traffic to a member, remove it from the pool.
Routing strategies
The strategy chooses the primary and orders failover candidates: priority_weighted, round_robin, or least_loaded. Strategy scope (workspace or caller) controls whether round-robin and least-loaded state is shared. Catalog is on the Pools reference.
Failover
Failover limits how many additional members Dome may try after the primary fails. Dome authorizes the primary first, then re-authorizes each candidate before trying it. Streaming can fail over only before the first response byte reaches the caller. Limits and behavior are on the Pools reference.
Pool resolution
When a request reaches a Gateway, Dome resolves the model value in a fixed order: exact pool name, exact connection name, upstream model id match, first matching match_when pool, then the workspace default pool. Exact names beat conditional routes. The match_when dialect is on the Pools reference. Routing does not replace authorization. Rules still evaluate the pool and selected model.
Response caching
Pool records store cache TTL and scope fields intended for identical-request reuse.
The gateway does not read these settings during model dispatch. Setting cache_ttl_secs or cache_scope does not cache responses.
Next steps
With that model-pool routing model in mind, continue with:
- Resources concept for how pools sit among resource types
- Models concept for the connections pools select
- LLM gateway concept for dispatch and failover on the wire
- Quotas concept for spend caps and pool spillover
- Pools reference for strategies, failover, and
match_when
- Pools to create pools, manage members, and attach Gateways
