AWS AgentCore Makes AI Agents Safe Enough for Real Work, and Stores Can Use That

A widely-cited industry report found that 95 percent of enterprise AI pilots fail to make it out of the testing phase. The report pointed to two root causes: trust and control. Companies could see that the agent was capable. They could not see whether it would stay within bounds when things got complicated, or whether it would perform consistently over time rather than brilliantly in the demo and unpredictably in production. AWS AgentCore is a direct answer to both of those problems.

The platform adds three layers that most agent frameworks leave out: policy enforcement that runs before any tool call, evaluation infrastructure that gives you a scored baseline from day one, and episodic memory that lets the agent learn from what worked and what did not across many real interactions. Together they shift the question from "can this agent answer questions" to "can this agent be trusted to run a real business process."

Write the policy rules in plain language before you build anything else

The policy layer is the most important piece to get right before anything else exists, because building an agent without guardrails is building a system you will have to retrofit safety into later. Retrofitting safety is always harder and usually incomplete.

AgentCore takes policies written in plain language and converts them into enforceable code automatically. You do not write regex filters or manual permission checks. You write rules the way a manager would write them in a policy document: "Only users with a verified purchase in the last 90 days can request a refund. Only users with an admin role can access order history for other accounts. Never provide a discount code unless the user has contacted support at least once." The system turns those statements into checks that run before any tool call is executed.

This plain-language input is important for a practical reason. The person who understands what the rules should be is often not a developer. A customer service manager knows that refunds over a certain dollar amount need human approval. A compliance officer knows which data fields cannot be shown to certain user classes. Writing the rules in plain language means those people can participate directly in defining what the agent is and is not allowed to do, and the developer's job is to connect the policy engine rather than to encode every business rule manually.

The policy engine itself runs in milliseconds and handles thousands of requests per second, which means it does not become a bottleneck as the agent scales. Every tool call -- whether that is querying a database, sending a notification, writing a record, or calling an external API -- is checked against the policy set before it executes. The agent cannot sidestep the policy by reasoning around it. The enforcement is at the execution layer, not the generation layer.

Start here. Before you build the first tool, before you write the first system prompt, write out the policy rules for what the agent can and cannot do. If you are building an agent that handles customer questions coming in from Facebook and Instagram ad campaigns, you know from the first day that it should not be able to access billing information without the right user context, and should not be able to modify order status without a verification step. Write those rules down in plain language. They become the foundation everything else is built on.

Route every agent request through the permission gateway

Once the policy rules exist, every agent request must flow through the permission gateway. This is the step developers sometimes try to shortcut by building the agent logic first and adding permissions later. That path creates problems because the agent's behavior shapes user expectations. If users learn to ask for things the agent should not provide and the agent provides them during development, removing that capability after the fact is disruptive in ways that a policy-first approach avoids entirely.

The permission gateway checks two things on every request: whether the user has the role required to take the action, and whether the action itself is within the bounds of the policy rules. Both checks must pass. Neither one alone is sufficient. A user with an admin role can still be blocked from taking an action the policy rules prohibit for all users. A user asking for something the rules allow can still be blocked if they do not have the right role.

This two-check structure is what creates the precision that makes compliance teams comfortable. They are not relying on the model to refuse forbidden requests; they are relying on an enforcement layer that the model does not control. The model can be instructed to be helpful and still be constrained to only the helpfulness that is authorized. Those two things are not in tension when the architecture separates them properly.

For a business integrating this with its CRM and website stack, the permission gateway also becomes the single authoritative place where access logic lives. When the access rules change, you change the policy document, not the agent's code. When you want to audit what the agent was and was not allowed to do during a given period, the policy logs give you that history.

Route every request through the gateway from the first day of development, and run the agent against the full policy set in testing before it handles any real user interaction.

Score the agent on a real eval set before it ever talks to a customer

An agent with good guardrails can still give unhelpful, inconsistent, or inaccurate answers within those guardrails. Policy enforcement and quality are separate problems. The evaluation infrastructure in AgentCore addresses quality.

The evaluation framework scores agents on five dimensions: correctness (is the factual content accurate), helpfulness (does the answer actually serve the user's need), faithfulness (is the answer grounded in the information the agent was given rather than invented), instruction-following (did the agent do what it was asked to do rather than what it thought was best), and relevance (is the response focused on what was asked or does it wander into tangential territory).

You build an eval set before deployment. An eval set is a collection of real or realistic questions paired with what the right answer looks like. It should reflect the actual distribution of questions your agent will receive -- not just the easy ones where any approach succeeds, but the edge cases, the ambiguous ones, and the ones where the right answer requires synthesizing information from multiple sources.

Running the agent against the eval set gives you a baseline score for each dimension. That baseline is what makes future changes legible. When you adjust the system prompt to improve faithfulness and the score goes up by twelve points, you know the change helped. When you add a new data source and the relevance score goes down, you know the integration introduced noise that needs to be filtered. Without the baseline you are guessing.

AgentCore can run evaluations continuously in production, not just before deployment. This catches drift: situations where the agent was performing well in testing and at launch but begins to degrade as the real question distribution turns out to differ from the eval set. Continuous evaluation also lets you run A/B tests on system prompt changes with confidence, because you are measuring real impact against a consistent scoring framework rather than relying on subjective impressions.

Set up the eval set before the agent goes live, and track the scores across every major change. The scores are the evidence that the agent is getting better rather than your belief that it is.

Add episodic memory only after guardrails and scores are solid

Episodic memory is the feature that makes the agent improve over time rather than performing identically on the thousandth interaction as it did on the first. It stores records of past interactions, including what happened, what the agent did, and whether the outcome was successful, and makes those records available to inform future responses.

The practical impact is significant. An agent with episodic memory can recognize that a particular type of question almost always leads to a misunderstanding in the third exchange and front-load the clarification instead. It can identify that users who come in from Google Ads campaigns for a specific service tend to have questions about pricing before they care about features, and reorder its approach accordingly. It can apply the lesson from a failed interaction -- where a technically accurate answer led the user to take an action they should not have taken -- to phrase similar answers more carefully in the future.

The memory spreads across the system rather than existing only within one conversation thread. That is what makes it organizational learning rather than just individual session context. When the agent learns something useful from one user's interaction, that knowledge is available to improve the next user's experience.

Adding episodic memory before guardrails and evaluations are solid is a mistake because it amplifies whatever patterns already exist in the agent's behavior. Good patterns become consistently good; bad patterns become entrenched. Get the policy layer right so that the behaviors being reinforced are the ones you want, get the eval scores to a solid baseline so you know what "good" looks like, and then add episodic memory to compound those good behaviors over time.

The evaluation framework can also measure whether episodic memory is actually improving performance over time. This is important because memory that does not improve scores is memory that is creating noise. Not every pattern the agent sees is a useful one to store. Regular evaluation helps you distinguish signal from noise in what the memory layer is learning.

An e-commerce store support agent built with all three layers

Consider what this looks like for a real use case: a mid-size e-commerce store that runs significant traffic from paid ads and wants to deploy an agent that handles the most common support questions, from order status and return policies to product questions and loyalty point redemption.

The policy layer handles the access rules. The agent can look up an order if the user can provide the order number and email address. It cannot look up orders for a different account. It can initiate a return for orders within the return window. It cannot approve returns for items outside the return window; those route to a human. It can provide loyalty point balances. It cannot adjust point balances; that requires staff access. These rules go into the policy document in plain language, and the permission gateway enforces them on every request.

The eval set covers the full range of real questions. Order status queries at various stages of fulfillment. Return requests for items inside and outside the return window. Questions about whether a specific product is compatible with a specific use case. Questions about loyalty tier requirements. The agent runs against this set before it handles any real customers, and the scores establish the baseline.

Episodic memory starts accumulating from the first real interactions. The agent notices that users who ask about return windows in the first message almost always follow up with a specific order number; it learns to ask for the order number proactively. It notices that questions about product compatibility for a particular product category have a high rate of follow-up questions about a specific feature; it learns to address that feature in the initial answer.

Over three months, the eval scores on helpfulness and instruction-following climb meaningfully. The rate of conversations that escalate to human agents drops. The agent is handling more questions correctly on the first response. None of that improvement required rewriting the agent or adding new capabilities. It came from memory learning what good looks like and applying that learning consistently.

This is what an enterprise-ready agent looks like. Not just capable, but trustworthy. Not just functional in the demo, but improvable in production. That is what closes the 95 percent gap.