Don't Build an Agent. Build the Agent Factory.

Published on the TEN INVENT blog · August 2026

Anyone can build an AI agent in a weekend. A good model, a few tools, a decent system prompt — and you have a demo that impresses in Monday's meeting. I've already written about the difference between a model and an agent and about what agents look like in production, so I won't repeat that part.

The real problem shows up at the second question, the one the weekend demo never touches: what happens when ten teams each want their own agent? Each with access to the CRM, to tickets, to databases. Each with their own API keys, their own prompts, their own idea of what "it works" means. Without an answer to that question you get what I now see in many companies: a pile of pilot agents that never reach production, because nobody can say who is allowed to touch what, how well anything actually performs, and what breaks on the next release.

The answer to it isn't "one more agent" — it's an agent architecture: the internal factory through which any team can ship governed, tested agents with controlled access to company systems. This article walks through the four pieces of that architecture and why you need each one.

The two layers

First clarification, because this is where most discussions get tangled: the agent the user sees and the platform underneath are different layers with different problems.

The application layer is what you build for one concrete use case: orchestration with sub-agents, RAG over the relevant documents, approval gates where a human validates sensitive actions. It's the layer we build in client projects too, and the one 95% of agent articles are about.

The platform layer is what makes the second, fifth and tenth agent not start from zero. It answers three questions the application can't solve on its own: who is allowed to reach which system, how good an agent is before you ship it, and how team B finds out that team A's agent already exists.

The rest of the article takes the bottom layer apart piece by piece.

The agent catalog: registry plus evaluations, not a wiki

The first piece sounds mundane and is the most underrated: an internal agent registry. Every agent has an entry with a description, the tools it can reach, the owning team and the current version. Teams check it before building something new — exactly like a service registry or an internal API catalog.

The difference between this and a wiki is the last two rows. A catalog without evaluations is a list of promises; a catalog with evaluations is a contract. Every agent comes with test sets — real input scenarios with expected outcomes — and scores along three axes: quality (how useful the answers are), correctness (how many claims are factually true, how many actions are the right ones) and safety (it refuses what it must refuse, doesn't leak data between customers, doesn't act outside its mandate).

The rule that ties it all together: you don't publish a new version to the catalog unless it passes the evaluations. Including — especially — the old suite. An agent that solves one new case at v1.4 but breaks three cases that worked at v1.3 does not ship. That's CI/CD, applied to agents:

The non-deterministic part is the only genuinely new thing. A code test is binary; an agent answer is "good enough" or not. That's why evaluations combine exact checks (did it call the right tool? did it respect the format?) with LLM-as-judge — a model that scores the answer against a rubric written by humans. Not perfect, but consistent, cheap, and it runs on every release, which a human review cannot.

On AWS this piece exists as a service — the evaluations in Amazon Bedrock AgentCore — but it builds just as well as your own pipeline: a set of scenarios in git, a job that runs the agent through them, and a judge that scores. The gate is what matters, not the tool.

The gateway: one place through which agents touch the world

The second piece solves an arithmetic problem. Three agents talking directly to five systems means fifteen integrations: fifteen places managing credentials, fifteen places to audit, fifteen implementations of rate limiting — usually zero. Every new agent multiplies the problem.

The gateway turns N × M into N + M: agents talk to a single layer, and that layer knows who is allowed to reach what.

Concretely, the gateway centralizes four things that every team would otherwise reimplement badly and differently: authentication (agents get identities, not API keys pasted into prompts), authorization (the support agent sees tickets, not salaries), rate limiting (an agent stuck in a loop doesn't bring the CRM to its knees) and logging (every tool call, with the agent's identity attached — the only way you'll ever answer "who changed this record?").

Its second function is less obvious but enormous in practice: the gateway translates existing APIs into MCP tools without rewriting them. You have an internal REST API or a Lambda function? Describe it at the gateway level and it becomes a tool any compatible agent can call. On AWS that's exactly what AgentCore Gateway does; the same pattern builds fine standalone.

MCP servers: the layer that actually exposes the tools

Below the gateway sit the servers that actually expose the tools and the data: CRM, tickets, documents, databases. I've written about MCP from a local-setup perspective; at enterprise level the logic is identical, only the stakes are higher — MCP is what stops the platform from becoming a pile of proprietary integrations.

Because the protocol is standard, the MCP server for tickets written by one team is reusable by any agent in the company, regardless of the framework the agent is built with. And when your vendors expose MCP servers themselves — increasingly common — you put them behind the same gateway and they get the same access rules and the same audit trail as the internal tools. The gateway + MCP pair is what turns "integration" from a three-week project into a configuration entry.

RAG: shared, not reinvented per agent

The last piece is the one you already know — with one positioning correction. RAG shows up in both layers, but each layer builds something different: the application decides what to ask and what to do with the answer; the platform owns the pipes — document ingestion, indexing, embeddings, and above all permission-aware filtering at retrieval time. If every team builds its own ingestion pipeline, you get five indexes aging at different speeds and at least one serving confidential documents to the wrong audience. Retrieval is infrastructure, like the database; the question of "which documents is this agent allowed to see" belongs to the platform, not to the good intentions of each prompt.

Why you need an agent architecture

Because none of the problems above can be solved inside an agent. You can write the best agent in the world and still have no answer to "who is allowed to reach what", "how good is it before release" and "what breaks on the next one" — those are properties of the system around the agents, not of any single agent.

And if you look at the four pieces together, you'll notice none of them is about AI in the strict sense. The catalog is governance. The evaluations are QA. The gateway is security and networking. Shared RAG is a data platform. The model — the "AI" part — is a dependency, not the product. That's why this is an architecture problem, not a better-prompts problem.

That, I think, is the most useful lens for this moment: with agents, we are where software was before DevOps. Everybody knows how to write one; almost nobody has the pipeline that ships it safely, measures it and retires it when it degrades. The gap between demo and production isn't a better model — it's the architecture around it.

And the practical order, if you start from zero, is the reverse of the enthusiasm: first a gateway with a single MCP server behind it and logging on every call. Then an eval set for the first agent — even 30 scenarios beat zero. The catalog comes third, once there's something to put in it. Shared RAG, when the second team asks for the same documents as the first.

An agent that works in a demo is a weekend. A factory through which ten teams ship agents without setting each other on fire is an architecture. If you're at the point where the first exists and the second doesn't, let's talk.