Enterprises spent the last decade building API programs. A large organization now runs thousands of internal APIs, each with a spec, owner, authentication scheme, and access policies enforced at a gateway. As agents move into production, those APIs are the obvious source of the context agents need: customer records, order status, inventory, pricing, and support tickets.
The more difficult question now is how agents reach those systems at scale. The Model Context Protocol (MCP) gave the industry a standard way to connect agents to tools. But the most common way teams expose APIs over MCP, one tool per API operation, starts to fail well before an agent touches a meaningful share of the enterprise’s API estate.
Here’s why that happens, what a better pattern looks like, and how platform teams can build a context layer that reuses the governance they already have.
Why one tool per endpoint doesn’t scale
The default way to turn an API into an MCP server is to take its OpenAPI spec and generate one tool per operation. That works for a small API. However, a typical enterprise CRM or ERP API can have hundreds of operations, and an agent that needs a few of those APIs quickly faces hundreds of tool definitions.
Using this approach, three things go sideways fast.
- Cost. Every tool definition loads into the model’s context before the agent does any work, and every intermediate result flows back through the model. A routine workflow, such as looking up a customer, pulling their open orders, and checking shipment status, pays for each hop in tokens.
- Accuracy. Once the tool list grows past what the model can reason about, tool selection degrades. The agent picks the wrong tool from a set of similar-looking options, or makes three calls where one would do.
- Latency. More context to process and more back-and-forth between the model and its tools slows the agent down. Users notice that long before anyone sees the token bill.
Teams usually respond by curating. They trim tool lists and split large APIs into many small MCP servers. That helps, but now every agent use case needs its own hand-maintained integration. That creates the same integration sprawl that API programs were built to eliminate.
Code mode: Let the agent write code against the API
The industry has converged on a better pattern, often called “code mode.” A code-mode server exposes an API as a single SDK-like interface instead of a flat list of tools. The agent writes short code snippets against that interface, the code runs in a sandbox, and only the final result returns to the model’s context.
In practice, the pattern is search, understand, and execute. The agent searches the API surface for the operation it needs, pulls that operation’s schema on demand, and runs it. The full API stays available, and only the parts the agent actually uses reach its context.
This plays to what models already do well. Large language models have seen enormous amounts of code in training and comparatively little tool-calling syntax. At the SDK level, an agent can use a large API it was never trained on because it reads the interface the way a developer would.
The efficiency gains are substantial. Because the agent works in code, one snippet can chain several API calls with loops, conditionals, and error handling. As such, the multi-step workflow above takes fewer round trips. Intermediate results stay out of the context window. In our testing, a workflow that consumed roughly 120,000 tokens through standard tool calling used about 1,200 in code mode.
To be clear, code mode carries its own cost. Running agent-written code means you need both a sandbox you trust and full observability into what the code actually called. A platform team adopting the pattern should treat both as requirements from day one.
Route agent traffic through the gateway you already run
Code mode solves the context window problem, but the governance problem remains. An agent calling a production API needs everything any other client needs: authentication, authorization, rate limiting, and audit logging.
Many teams building agent integrations recreate these controls one MCP server at a time. Credentials end up in server configs. Access rules get redefined by whoever wrote the integration. Agent traffic bypasses the gateway, so it never shows up in the logs security teams already watch.
A better approach treats the context layer as an extension of the API platform. The spec, schemas, authentication requirements, and access policies for each API already live where the API is managed. Generate the agent-facing interface from that source of truth, and route agent calls through the same gateway that governs every other consumer. Existing policies then apply to agents automatically, and the agent becomes one more governed client.
Discovery matters too. Register each agent-ready interface in a central catalog, so the next team building an agent can find and reuse it instead of building a fourth integration to the same CRM.
Compose one context surface per agent
APIs are only part of the picture. Most enterprises already run MCP servers, some from SaaS vendors and some built by internal teams. Agents also need access to enterprise data sources. A useful agent draws on all of them.
A context layer should let platform teams compose sources. Take several internal APIs and a few existing MCP servers, and expose them together as one code-mode interface scoped to a specific agent’s job. A customer support agent gets ticketing, CRM, and shipping. It does not get payroll.
That scoping does two jobs at once. It is a security control, because the agent can reach only what its task requires. It is also a performance control, because a smaller, relevant interface keeps the model focused.
Curation is the point. More context is not automatically better context. The goal is to make the breadth of the enterprise available to agents while keeping what reaches the model small and relevant.
How we implemented code mode at Kong
At Kong, we built these principles into Kong Context Mesh, now generally available as part of Kong AI Management. They extend the approach we introduced with Kong Context Mesh earlier in 2026.
Platform teams bring together their APIs, existing MCP servers, and enterprise data sources, and Kong AI Gateway generates the code-mode MCP interfaces agents use to reach them. Because the gateway generates and serves those interfaces, agent access sits on the same platform teams already use to govern their API and AI traffic.
Agents are only as good as the context they can reach. Effective agents need broad access to enterprise systems, delivered selectively and on demand. The fastest-moving enterprises are treating that context as infrastructure. They will generate it from systems they already manage, govern it with policies they already enforce, and scope it to the job each agent needs to do.
APIs became the backbone of digital business because platform teams made them discoverable, secure, and governed. Agents need the same discipline. The good news is that most enterprises have already built the foundation.
—
New Tech Forum provides a venue for technology leaders—including vendors and other outside contributors—to explore and discuss emerging enterprise technology in unprecedented depth and breadth. The selection is subjective, based on our pick of the technologies we believe to be important and of greatest interest to InfoWorld readers. InfoWorld does not accept marketing collateral for publication and reserves the right to edit all contributed content. Send all inquiries to doug_dineley@foundryco.com.
