Call a model through a pool

Tool calls are one half of what an agent does. The other half is inference, and it carries its own problems: a provider key sitting in an environment variable, spend nobody notices until the invoice, and no record of which agent asked for what. A model connection is one upstream LLM endpoint and its provider credential, for example Claude with your Anthropic key. A pool groups one or more of those connections behind a single name. The agent puts the pool name in the request’s model field. Dome chooses which connection serves the call. That split is what lets you swap providers, add failover, or change weights later without touching the app.

Hand this to an AI agent. It adds models, puts them in a pool, shows routing/failover and spend caps, and hands you the curl that proves the pool holds.

Open in Cursor

In this tutorial, you will put an LLM behind the same Gateway your tools already sit behind. The provider key stays in Dome, the agent names a pool rather than a vendor model, every call is attributed, and on Pro spend can be capped before it reaches the provider. You will also put two different models in that pool and switch which one serves traffic without changing the curl. To do this, you will:

Add a model connection

Put it in a pool

Create a pool, add the connection, and attach the pool to the gateway.

Cap the spend

Learn how pool quotas work. Set one on Pro, or skip on Free.

Authorize the pool

Deploy a rule that permits this pool and nothing else.

Point at the LLM ingress

Form the Gateway /v1/chat/completions URL and export credentials.

Call the pool

POST once naming employee-summary.

Route across two members

Add a second model, flip priority, and curl again with the same pool name.

Verify the results

Check the audit trail, and the spend if you set a quota.

Prerequisites

For this tutorial, you will need:

Govern your first agent completed, with its sandbox still active. This tutorial reuses that workspace, its Default gateway, and the hr-assistant agent.
An API key for OpenAI or Anthropic. Dome calls the provider on your behalf, so the spend lands on your provider account.
A second model for the routing section: another model on that same key, or a key for the other provider.
An agent token (dome_…) for hr-assistant, the key from the first tutorial, or mint another with dome agents create-key.

Optional: a Pro plan if you want to set a spend quota on the pool. Quotas are included on Pro. Free workspaces skip that step and continue with the rest of the tutorial.

Confirm the workspace before you start:

dome context current

The workspace should read sandbox-get-started. Switch back if it does not:

dome context use sandbox-get-started

This tutorial runs entirely in a sandbox. In a production workspace, adding a provider connection and setting spend caps are typically finance and operator actions, not developer ones.

Add a model connection

A model connection is a named route to one provider model, holding the credential Dome uses to reach it. Agents can call a connection by name, but this tutorial puts the connection in a pool instead. Naming the pool keeps the client’s model field stable when you later add failover, change weights, or swap the upstream provider.

OpenAI
Anthropic

dome model add openai-mini \
  --provider openai \
  --model gpt-4o-mini \
  --api-key "$OPENAI_API_KEY"

dome model add claude-sonnet \
  --provider anthropic \
  --model claude-3-5-sonnet-20241022 \
  --api-key "$ANTHROPIC_API_KEY"

The rest of this tutorial uses openai-mini. Substitute your connection name if you chose Anthropic. --api-key stores the credential in Dome’s secret store, which is why nothing downstream needs a copy. For providers with their own identity story, --use-workload-identity skips the stored credential and uses the gateway’s cloud-native chain instead, such as GCE metadata for Google or the AWS SDK chain for Bedrock. That avoids holding a long-lived key at all. Leave the connection off the Gateway for now. A connection that is not a pool member and not attached to a gateway is registered but unreachable. You store the credential first, then expose the model only through the pool in the next step, so the agent never learns a direct connection name to call around the pool. Confirm the connection is registered and the credential is stored:

dome model get openai-mini

Put it in a pool

A pool is the name your client will put in the model field. Dome picks which member serves each request, and can try another member when the first fails. Create employee-summary and attach it to Default:

dome model pool create employee-summary \
  --description "HR assistant employee summarization" \
  --routing-strategy priority_weighted \
  --failover-max all \
  --gateway Default

Setting	What it means here
`priority_weighted`	Prefer members by priority tier, then by weight within a tier. With one member, every call goes there.
`--failover-max all`	Walk every eligible member before giving up.

Then add the connection as its only member:

dome model pool member add employee-summary openai-mini \
  --priority 0 \
  --weight 1

--priority 0 marks this member as primary. Dome tries a --priority 1 member only if the primary fails. Even with one member, the pool still matters. The client and the rule both name employee-summary. After the first curl succeeds, you will add a second model and switch which member serves without changing either. Confirm the pool and its membership:

dome model pool get employee-summary
dome model pool member list employee-summary

Cap the spend

A quota is a dollar cap on a subject over a window. Enforcement happens before the provider is called, so an exhausted cap costs nothing. That is how you stop a runaway session from becoming an invoice surprise, and why quotas belong next to pools even when you are not setting one yet.

Cost quotas are included on the Pro plan. Free workspaces cannot create them (limit-max-cost-quotas is zero). On Free, read this section and continue to Authorize the pool. The rest of the tutorial does not depend on a quota.

On Pro, set five dollars per month on the pool so every member shares one budget:

dome model quota set \
  --subject pool \
  --pool employee-summary \
  --limit 5 \
  --window monthly \
  --name "sandbox cap"

Confirm it, along with what has been spent so far:

dome model quota list

The subject is what makes a quota useful. --subject pool caps everything flowing through this pool no matter which member served the call. --subject model caps one connection across every pool it belongs to. --subject agent caps one agent across every model. --subject act-as caps an individual end user, which is what you reach for when one person’s runaway session should not exhaust the team’s budget. Windows are daily or monthly, and a quota can be created with --disabled if you want it in place before you start enforcing it.

Authorize the pool

The Gateway attachment makes employee-summary reachable. It does not decide who may call it. Tool rules from earlier tutorials cover mcp:call, not inference. Inference is a separate action, llm:invoke, on Dome::LLMModel. Allow hr-assistant to call models in the employee-summary pool, and deny every other inference call. Scope the rule to the pool name rather than a connection like openai-mini, so a failover member you add later inherits the same permit. Create hr-assistant-llm.cedar:

hr-assistant-llm.cedar

permit(
  principal is Dome::Agent,
  action == Dome::Action::"llm:invoke",
  resource is Dome::LLMModel
) when {
  resource.pool == "employee-summary"
};

forbid(
  principal is Dome::Agent,
  action == Dome::Action::"llm:invoke",
  resource
) unless {
  resource has pool && resource.pool == "employee-summary"
};

resource.pool is how Cedar sees membership. Authorizing on Dome::LLMModel::"openai-mini" would lock the rule to one connection and miss any member you add later.

What authorizing the connection looks like

A connection-scoped rule permits only that one model. Adding a failover member later would still be denied until you rewrite the bundle:

permit(
  principal is Dome::Agent,
  action == Dome::Action::"llm:invoke",
  resource == Dome::LLMModel::"openai-mini"
);

forbid(
  principal is Dome::Agent,
  action == Dome::Action::"llm:invoke",
  resource
) unless {
  resource == Dome::LLMModel::"openai-mini"
};

Deploy it as its own agent-scoped bundle:

dome rules apply hr-assistant-llm.cedar --agent hr-assistant --name hr-assistant-llm

Check the decision before you write any code. Simulation runs the same evaluator as the gateway, with no call and no spend:

dome rules simulate --agent hr-assistant --action llm:invoke \
  --resource employee-summary --resource-type llm_pool

Expect ALLOW. The llm_pool resource type is how simulation addresses a pool the way a live call does: it stamps resource.pool so the permit above matches. Substituting a pool name you never created should return DENY.

Dome rewrites pool-name literals to pool UUIDs when the bundle deploys, so the rule you write with "employee-summary" is what the gateway evaluates against the unspoofable pool id. You author names. The platform stores ids.

Point at the LLM ingress

Point curl at the Default gateway’s LLM ingress and authenticate as the agent. That path is where OpenAI- and Anthropic-shaped model traffic enters the data plane:

https://<gateway-host>/gateways/<DEFAULT_GATEWAY_ID>/v1/chat/completions

The first tutorial used the same Gateway’s tools ingress at /mcp. Model traffic shares the /gateways/<id> prefix and uses /v1/... instead. Retrieve the three placeholders:

AGENT_API_KEY. The Token: dome_… value from Govern your first agent. Tokens are shown once and cannot be recovered. If you did not save it, mint another:
dome agents create-key hr-assistant --name pool-curl
GATEWAY_HOST. Prepend https:// to your current Dome host, for example https://gateway.dev.domesystems.ai.
dome context current
DEFAULT_GATEWAY_ID. The UUID of the Default gateway.
dome gateway list

The result should look similar to https://gateway.dev.domesystems.ai/gateways/3f9a2c14-8d7e-4b1a-9c02-5e6f7a8b9c01/v1/chat/completions. Export once. DOME_GATEWAY_URL stops at the Gateway. curl appends /v1/chat/completions:

export DOME_GATEWAY_URL="https://GATEWAY_HOST/gateways/DEFAULT_GATEWAY_ID"
export DOME_TOKEN="AGENT_API_KEY"

Call the pool

The Gateway speaks the same OpenAI chat-completions protocol. Compared with calling the provider directly, three fields change:

Field	Value
URL	`$DOME_GATEWAY_URL/v1/chat/completions`
Authorization	Agent bearer token (`dome_…`), not the provider key
`model`	`employee-summary` — the pool name, not `gpt-4o-mini` or `openai-mini`

Dome picks which member serves the call. The client never names the upstream model or connection. Call the pool once:

curl -sS -X POST "$DOME_GATEWAY_URL/v1/chat/completions" \
  -H "Authorization: Bearer $DOME_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "employee-summary",
    "messages": [
      {
        "role": "user",
        "content": "Summarize this employee in one sentence: E001 Alice Johnson, Senior Engineer in Engineering."
      }
    ]
  }'

Expect a normal chat-completions JSON body. The assistant text is a one-line employee summary. The important proof is that the call succeeded while naming employee-summary, not gpt-4o-mini or openai-mini. Leave the curl exports in your shell. The next section reuses them after you change pool membership. Provider SDKs work the same way later: OpenAI clients use base URL …/gateways/<id>/v1 and model: "employee-summary". Anthropic clients omit the trailing /v1 on the base URL because their SDK appends /v1/messages. This tutorial sticks to curl so the wire shape is visible without an SDK.

Troubleshooting

These are remediation steps if you get stuck:

400 with select a gateway. The URL is missing its /gateways/<id> segment. A bare gateway host serves no inference.
403 with model not available in this gateway. The pool exists but is not attached to Default, or it has no members. Confirm both:
dome gateway get Default dome model pool member list employee-summary

403 with a permission_error naming a rule. Cedar denied the call. Reproduce it without spending anything:

dome rules simulate --agent hr-assistant --action llm:invoke \
  --resource employee-summary --resource-type llm_pool

404 with no connection for requested model. You passed a provider model id such as gpt-4o-mini, or the connection name openai-mini, where Dome expects the pool name. Check what this workspace exposes:
dome model pool list
429 with llm: quota exceeded. The cap is working. The message names the subject that ran out. Review it, and raise it deliberately rather than by reflex:
dome model quota list
If you never set a quota (Free plan), this error will not appear from Dome. Check your provider account instead.
401 from the gateway. The bearer token is the provider’s key, not Dome’s. This endpoint authenticates the agent, and the provider credential never leaves the platform.

Route across two members

A single pool name can front more than one upstream. Add a second member for failover when the primary is down, shift traffic with weights, or promote a cheaper or stronger model without rewriting curl or Cedar. Clients and rules keep saying employee-summary. Membership decides who answers. In this section you add a failover member, then flip priorities so the new connection becomes primary. Failover is already set: priority_weighted with --failover-max all tries a --priority 1 member only if the primary fails.

Add the second connection

A pool can mix providers. The same employee-summary name can hold OpenAI and Anthropic members side by side. Pick same provider or other provider below.

Same provider
Other provider

Reuse the key you already stored. Match the tab to your primary connection:

OpenAI
Anthropic

If your primary was openai-mini, add a stronger OpenAI model:

dome model add openai-full \
  --provider openai \
  --model gpt-4o \
  --api-key "$OPENAI_API_KEY"

dome model pool member add employee-summary openai-full \
  --priority 1 \
  --weight 1

If your primary was claude-sonnet, add a lighter Anthropic model:

dome model add claude-haiku \
  --provider anthropic \
  --model claude-3-5-haiku-20241022 \
  --api-key "$ANTHROPIC_API_KEY"

dome model pool member add employee-summary claude-haiku \
  --priority 1 \
  --weight 1

Put a second vendor behind the same pool name. If your primary was OpenAI:

dome model add claude-sonnet \
  --provider anthropic \
  --model claude-3-5-sonnet-20241022 \
  --api-key "$ANTHROPIC_API_KEY"

dome model pool member add employee-summary claude-sonnet \
  --priority 1 \
  --weight 1

If your primary was Anthropic, add openai-mini the same way and attach it at --priority 1.

Confirm both members:

dome model pool member list employee-summary

You should see priority 0 (primary) and priority 1 (failover). Healthy calls still hit priority 0. The second member is not a second name for the app to learn. curl and Cedar keep saying employee-summary.

Switch the primary

Flip priorities so the second member becomes primary. The curl keeps the same model value, but Dome routes it to the new primary. Substitute the connection names you actually added:

# Example: openai-mini was primary, openai-full was failover
dome model pool member update employee-summary openai-mini \
  --priority 1 \
  --weight 1

dome model pool member update employee-summary openai-full \
  --priority 0 \
  --weight 1

dome model pool member list employee-summary

Call again with the same request, still model: "employee-summary":

curl -sS -X POST "$DOME_GATEWAY_URL/v1/chat/completions" \
  -H "Authorization: Bearer $DOME_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "employee-summary",
    "messages": [
      {
        "role": "user",
        "content": "Summarize this employee in one sentence: E001 Alice Johnson, Senior Engineer in Engineering."
      }
    ]
  }'

You now have one pool name and two backends. Membership and priority decide which serves. Keep the preferred model at priority 0 and the alternate at priority 1. With --failover-max all, Dome tries the backup when the primary fails. Flip priorities back if you want the original primary for Build a governed app. The application only names employee-summary. It does not care which member is on top.

Verify the results

Confirm the calls went through the pool and landed on hr-assistant, not a shared provider key. On Pro, also confirm they counted against your spend cap. Read the record:

dome audit query --limit 20

You should see llm.model_call.attempted and llm.model_call.completed for hr-assistant, carrying token usage and the member that served each call. The first curl and the post-switch curl should name different members if you flipped priority. Denials appear as access.denied, the same event type a rejected tool call produces, because both are authorization decisions. If you set a quota on Pro, check the spend. The used figure moves after each call:

dome model quota list

To confirm the cap enforces rather than merely reports, lower it below what you have already spent and call again:

dome model quota update <QUOTA_ID> --limit 0.01

The next request returns 429 and never reaches the provider. Restore the cap afterward with another update, or remove it:

dome model quota rm <QUOTA_ID>

Skip the quota commands on Free. There is nothing to list or update. The next tutorial wires the same pool into an application where the model and tool calls run together.

Next steps

You learned how to hold a provider key in a model connection, route through a pool with failover, authorize llm:invoke on the pool, and attribute spend. Keep the sandbox-get-started workspace for the rest of this track. Continue with:

Build a governed app to wire model and tool calls in one application
Pools for routing strategies and membership
Set Usage Limits for spend caps on pools, models, and agents

Get Started

Role

Production

Examples

Call a model through a pool

Add a model connection

Put it in a pool

Cap the spend

Authorize the pool

Point at the LLM ingress

Call the pool

Route across two members

Verify the results

Prerequisites

Add a model connection

Put it in a pool

Cap the spend

Authorize the pool

Point at the LLM ingress

Call the pool

Route across two members

Add the second connection

Switch the primary

Verify the results

Next steps

Add a model connection

Put it in a pool

Cap the spend

Authorize the pool

Point at the LLM ingress

Call the pool

Route across two members

Verify the results

​Prerequisites

​Add a model connection

​Put it in a pool

​Cap the spend

​Authorize the pool

​Point at the LLM ingress

​Call the pool

​Route across two members

​Add the second connection

​Switch the primary

​Verify the results

​Next steps

Prerequisites

Add a model connection

Put it in a pool

Cap the spend

Authorize the pool

Point at the LLM ingress

Call the pool

Route across two members

Add the second connection

Switch the primary

Verify the results

Next steps