Agentic AI on a budget: a reference architecture for a small business

Most small businesses that ask me about “agentic AI” don’t need a platform. They need one agent that does one job well, and a bill that never surprises them. This is the design I’d start with on AWS, with the arithmetic shown so you can redo it for your own numbers.

The short version: for a business handling 3,000 customer conversations a month, the whole thing runs for roughly $30 to $35 a month, about a cent per conversation. The way to keep it there is mostly model choice and a few hard limits, not clever infrastructure.

The job

Picture a 15-person online business. Its support inbox gets about 3,000 conversations a month: “where is my order”, “can I return this”, “do you ship to Canada”, plus the occasional angry customer.

The agent’s job is narrow:

1. Read the incoming message.
2. Look up the order if there is one.
3. Answer policy questions from the company’s own documents.
4. Draft a reply. A person reads it and clicks send.

That last step matters more than it looks. Keeping a human on the send button keeps the risk low while you learn how the agent behaves. It also gives you evaluation data for free: every edit someone makes to a draft tells you where the agent is weak.

The architecture

Each message takes the same path:

1. Input guardrail. Amazon Bedrock Guardrails screens the customer’s text for abuse, prompt-injection attempts and topics you’ve ruled out.
2. Triage with the smallest model. Amazon Nova Micro classifies the message: intent, language, spam, and whether it needs the agent at all. A plain “where’s my order” can go straight to the orders tool. Triage also decides whether this conversation deserves the bigger model.
3. The agent loop, in Lambda. Claude Haiku 5.5 handles most conversations. Claude Sonnet 5.5 takes the ones triage flags as complex, sensitive or high-value. The loop keeps its state in a DynamoDB table.
4. Tools. Two of them: a knowledge base over the company’s docs, with vectors in Amazon S3 Vectors, and a small function that calls the orders API.
5. Output guardrail, then a draft. The final reply is checked once more and lands in the helpdesk as a draft.

Everything except the model calls is serverless and scales to zero. Nothing runs, and nothing bills, between messages.

Lambda is the right home for the loop at this size. If your agent’s sessions get long, or you need stronger isolation between sessions, Amazon Bedrock AgentCore Runtime is the next step up. It bills CPU only while the agent is actually working, not while it waits on a model or a tool.

What it costs

These are the assumptions behind the numbers. Change them to match your business.

• 3,000 conversations a month, about 4 model calls each
• Each call sends a 3,000-token fixed prefix (instructions and tool definitions) plus 2,500 tokens of conversation and retrieved text, and gets back about 300 tokens
• 15% of conversations escalate to Sonnet
• 90% of calls reuse the cached prefix
• Guardrails check the customer’s input and the final reply, with two policies (content filters and denied topics)
• About 2,000 pages of documents, split into 10,000 chunks

Line item	Per month
Triage, Nova Micro	$0.09
Agent turns, Haiku 5.5	$4.74
Escalations, Sonnet 5.5	$16.24
Guardrails, input and final reply	$6.30
S3 Vectors storage and queries	$0.02
Lambda, API Gateway, DynamoDB	under $2
CloudWatch logs and metrics	$1 to $5, depending on what you log
Total	about $30 to $35

Embedding the documents in the first place is a one-off cost of about three cents with Titan Text Embeddings V2.

Two things in that table surprise people.

Escalations are 15% of the conversations and about 77% of the model spend. The escalation rule is the most important cost decision in the design, so write it down and watch it.

The guardrails cost more than the default model. That’s not a reason to drop them. It is a reason to check at the edges instead of on every intermediate step, which in this setup would push guardrail spend from $6.30 to about $10.80.

Five levers that matter

1. Make the small model the default

Run the same workload on Sonnet for every call, with no caching, and the model bill is about $168 a month instead of about $21. That’s 87% saved just by routing. Start small, measure where the small model fails, and escalate only those cases.

2. Cache the fixed prefix

Your instructions and tool definitions are identical on every call, so put them first in the prompt and turn on prompt caching. A cache read costs a small fraction of normal input: for Haiku 5.5, $0.01 per million tokens instead of $0.10. The only rule is ordering: everything that never changes goes before everything that does.

3. Keep the context lean

With caching on, the cost that remains is the part of the prompt that changes: conversation history and retrieved text. That’s more than half of the Haiku line above. Retrieve three to five good chunks instead of ten mediocre ones. Once a conversation runs past a handful of turns, summarize its history instead of resending all of it.

4. Check at the edges

Guardrails are billed per 1,000 characters, per policy. Screen what comes in and what goes out. The model’s private reasoning between tool calls doesn’t need its own inspection.

5. Prefer storage that’s free when idle

For a corpus this size, S3 Vectors costs pennies: you pay per gigabyte stored and per query. Before choosing any vector store, check whether it bills by the hour while nobody is asking it anything. At small-business volume, an idle floor can easily be the biggest line on the bill.

Put a ceiling on it

The table above is the expected case. The expensive case is an agent that gets stuck.

Say a confused agent loops for 25 steps on Sonnet, re-reading a growing context each time. That’s about $1 for one conversation. If 5% of conversations went that way, you’d add $150 a month, roughly five times the cost of the whole system. Agents fail like this quietly, so the limits have to be built in from the start:

• Hard caps. A maximum of 8 steps per conversation, a maximum output length per call, and a total token budget per conversation. When a cap is hit, the agent stops and hands the conversation to a person.
• Cost per conversation as a metric. Multiply the tokens used by the price and send the result to CloudWatch with every conversation. Alarm on the 95th percentile, not the average.
• An escalation-rate alarm. If more than about 25% of a day’s conversations go to Sonnet, triage has drifted. You want to hear that from an alarm, not from the invoice.
• AWS Budgets. Set a monthly figure with alerts at 50%, 80% and 100%. Bedrock usage shows up on your normal AWS bill, so Budgets sees it like everything else.

When to step up

This design stretches further than you’d think. At ten times the volume, it still costs a few hundred dollars a month. Some signs it’s time to change something:

• Work that can wait, like overnight summaries or bulk tagging of old tickets, can move to batch inference, which is billed at half the on-demand price.
• Long-running or multi-step workflows, or memory that has to persist across sessions, point toward AgentCore’s Runtime, Memory and Gateway. They’re priced per use, so they suit this same budget mindset.
• Strict data-residency rules mean regional endpoints instead of global ones. For the newer Claude models on Bedrock, that adds 10% to the token price.
• Drafts going out without edits. Once a category of drafts is consistently sent unchanged, consider letting that category send on its own, and keep watching the numbers.

A starting checklist

• One job, written down, with a clear definition of a good outcome
• A person approves every outgoing message at first
• The small model by default, with the escalation rule written down
• The fixed prompt prefix cached
• Caps on steps, tokens per call and tokens per conversation
• Guardrails on input and final output
• Cost per conversation tracked as a metric, with an alarm
• An AWS Budget with alerts

None of this requires a platform team. It requires deciding what the agent is for, and refusing to let it spend money you haven’t agreed to.

---

Prices are the published US rates as of October 2026. Claude prices are for global endpoints and prompts up to 100,000 tokens. Prices change often, so check the current pages before you budget: Amazon Bedrock pricing, Claude pricing, Amazon S3 pricing (S3 Vectors), AgentCore pricing.