AgentCore Gateway Adds InvokeModel and Model Fallback: What Architects Should Know
Prompt & Circuit Editorial
October 23, 2026 · 12 Min Read

Why this matters
Most agent platforms eventually need a policy for what happens when the preferred model is throttled or unavailable. On Oct 22, 2026, AWS announced that Amazon Bedrock AgentCore Gateway supports the Amazon Bedrock InvokeModel and InvokeModelWithResponseStream APIs on its inference endpoint, along with model fallback routing for inference traffic [1]. AWS says the gateway does this so agents stay available without each client implementing its own retry logic [1].
What the gateway now does
AgentCore Gateway is described as a fully managed AI gateway that provides a single, secure entry point for agentic traffic, connecting agents to tools, to other agents, and to large language models [3]. A gateway can have multiple targets across three categories: MCP, HTTP, and inference [2]. Inference targets route LLM requests to one or more model providers through a unified, model-based routing endpoint, and the gateway selects the destination based on the model field in the request [2]. The documentation names Amazon Bedrock, OpenAI, and Anthropic as example providers [2].
With InvokeModel support, you can point an existing Amazon Bedrock SDK integration at the gateway and reach any model provider through one governed endpoint, without rewriting client code for each provider [1].
How fallback is configured
Fallback is defined in gateway rules. You define an ordered chain of alternative destinations in a gateway rule, along with the conditions that trigger a switch, such as throttling, server errors, or response timeouts [1]. The gateway then routes a request to an alternative model or provider when the primary destination is throttled, out of capacity, or unreachable [1]. A built-in circuit breaker skips destinations that are currently failing, which avoids added latency for failed attempts [1]. From the caller's side, you send one request and receive one response, regardless of the internal trial and error [1].
The rules documentation adds the mechanics [5]:
- Rules have a priority from 1 to 1,000,000, and lower numbers mean higher precedence [5].
- A matchModels condition matches inference requests based on the model the caller requested, and it accepts a model ID or glob pattern such as gpt-5.* [5].
- Use matchModels with a routeToTarget action to pin which target serves a model, then add a fallback block to turn the route into a fallback chain [5].
- For inference targets, a staticRoute, and each weightedRoute leg, can pin the served model and carry an optional fallback block [5].
One detail deserves attention. The default operator for matchModels is StringLike, so wildcards stay active, whereas matchPrincipals defaults to StringEquals [5]. Use StringEquals when you want an exact, literal model match [5].
Limits to design around
The rules page lists hard limits that shape a fallback design [5]:
| Limit | Value |
|---|---|
| Rules per gateway | 20 |
| Maximum conditions per rule | 3, at most one of each type |
| Maximum actions per rule | 2 |
| fallback.targetFallback entries | 1 to 2 |
| fallback.triggerOn.statusCodes entries | 1 to 10 |
| fallback.triggerOn.errors entries | 1 to 3 |
| firstByteTimeoutMilliseconds range | 50 to 900,000 |
Two consequences follow. First, a fallback chain holds one or two alternatives after the primary, so this is not a mechanism for long cascades [5]. Second, with 20 rules per gateway, a large catalogue of models needs glob patterns rather than one rule per model [5]. Weighted routes split traffic between exactly two targets, with weights from 1 to 99 [5].
The announcement states that the features are available in all AWS Regions where AgentCore Gateway is available, except those in China [1]. It does not state a preview status, pricing or a quota for the new capability, so check the current User Guide before committing to a design [1].
Gateway fallback versus cross-Region inference
Bedrock already offers cross-Region inference, and the two solve different problems. Cross-Region inference works through inference profiles, which define a foundation model and the AWS Regions to which requests can be routed [6]. A geographic profile keeps processing within a geography such as the US or EU, while a global profile routes across all supported commercial Regions [6]. AWS states that to meet an availability target you should add bounded retries and multi-Region or multi-model fallback in your application [6].
The AWS Well-Architected Agentic AI Lens takes the same line. It recommends implementing model tier fallback using Bedrock cross-Region inference for availability, substituting alternative models when the primary is degraded [4]. It also advises logging the retrieval tier and model tier used in each invocation, so you can see which tier produced an answer and whether the fallback path is taken more often than expected [4].
| Concern | Cross-Region inference | Gateway model fallback |
|---|---|---|
| What changes on failure | The Region that processes the request [6] | The model or provider that serves the request [1] |
| Where it is configured | An inference profile [6] | A gateway rule [5] |
| Data residency | Geographic profiles keep processing within a geography [6] | Depends on the destinations you list in the chain |
| Client changes | Choose an inference profile [6] | Point the Bedrock SDK at the gateway [1] |
The two are not mutually exclusive. Cross-Region inference widens the pool of compute behind one model, while gateway fallback moves to a different model or provider when that pool is not enough.
Governance and data-handling implications
Because the gateway sits in front of several providers, a fallback can send a prompt to a destination the primary route would never use. The documentation says the gateway selects the destination from the model field and can reach providers such as OpenAI and Anthropic [2]. Treat every entry in a fallback chain as an approved processing location for the same data. The sources do not describe data-handling terms for individual providers, so verify those separately.
Gateway authorisation also matters. Each gateway must have an inbound authorization configuration, with OAuth (JWT), IAM (AWS Signature Version 4), authenticate only, or no authorization for development and testing [2]. Production inference endpoints should use one of the first two.
Architect's checklist
- Decide which models are acceptable substitutes for each primary, and confirm that each one is approved for the data it may receive.
- Keep fallback chains short: the limit is 1 to 2 alternatives per fallback block [5].
- Choose trigger conditions deliberately, from throttling, server errors, or response timeouts [1], and set the first-byte timeout within the 50 to 900,000 millisecond range [5].
- Use StringEquals on matchModels when a rule must match one exact model [5].
- Leave gaps between rule priorities, as the documentation advises, so you can insert rules later [5].
- Record which model tier served each request so you can see how often the fallback path is used [4].
- Combine gateway fallback with cross-Region inference where Regional capacity is also a concern, and keep bounded retries in the application [6].
- Pick an inbound authorizer that fits production, not the development-only option [2].
- Re-check the User Guide for pricing and quota details, because the announcement does not state them [1].
Sources
Numbers in square brackets refer to these sources, checked on 2026-10-23.
- AgentCore Gateway now supports Bedrock InvokeModel APIs and model fallback - AWS
- Core concepts for Amazon Bedrock AgentCore Gateway - Amazon Bedrock AgentCore
- Amazon Bedrock AgentCore Gateway: A secure AI gateway for agents, tools, and models - Amazon Bedrock AgentCore
- AGENTREL05-BP01 Design modular, fault-tolerant agentic reasoning components - Agentic AI Lens
- Add rules to a gateway - Amazon Bedrock AgentCore
- Route model inference requests across AWS Regions with cross-Region inference - Amazon Bedrock
Responses (0)
Sign in to join the conversation and share your thoughts.

Prompt & Circuit Editorial
Contributing Editor
Articles on AI infrastructure for cloud architects, written from official vendor sources and checked claim by claim before publishing.
Continue Reading

Keeping Amazon Bedrock Traffic Private with VPC Endpoints
Which Bedrock APIs can be reached through AWS PrivateLink, how private DNS changes your code, and why the default endpoint policy deserves attention.

Step Functions Workflows as AgentCore Gateway Tools: Where Agents Stop and State Machines Start
Amazon Bedrock AgentCore Gateway can now expose AWS Step Functions workflows to agents as MCP tools. Here is how pinning, Standard versus Express, and IAM shape the design.