What Building an Internal MCP Gateway Actually Teaches You About Enterprise Agent Readiness

Most engineering leaders who have read the Model Context Protocol specification come away thinking the hard part is behind them. The protocol is well-defined, the tooling ecosystem has matured considerably through 2025, and reference implementations are available for most major languages. What they discover once building starts is that MCP itself is not the constraint. The constraint is everything the gateway has to sit between: the sprawling, inconsistently documented, permission-layered surface area of enterprise SaaS, and the requirement that agents receive context that is actually good enough to produce decisions worth trusting.

Companion piece to our broader work on enterprise agent architecture. See The MCP Stack: How Engineering Teams Should Architect AI Agents That Stay Accurate as the World Changes for a technical guide to grounding agents in live, authoritative data sources and managing knowledge freshness tradeoffs.

The Gateway Is Not a Router, It Is a Trust Boundary

The first conceptual mistake teams make is treating the MCP gateway as a thin routing layer that passes tool calls through to downstream APIs. In a single-user, single-system prototype that framing is harmless. In an enterprise deployment it is dangerous.

The gateway is where authorization decisions happen. It is where the identity of the requesting agent, the identity of the human principal behind that agent session, and the permissions that principal holds in each connected system must be resolved and enforced before any tool call executes. Treating that as a downstream concern means your agent can, in principle, exfiltrate data from a SaaS system that the user has read access to but that your security team never intended to expose to an automated process.

Building Sierra, our internal agent platform, made this concrete very quickly. The gateway needed to maintain a per-session permission envelope that was derived from the authenticated user's entitlements in each connected system, not just a static service account with broad access. That is a materially different engineering problem from building a proxy.

Context Quality Is the Binding Constraint on Agent Performance

Once the access control architecture is sound, the next thing that becomes apparent is that agents produce mediocre outputs not because the model is weak but because the context passed to it is weak. This is the finding that most pilot teams miss because their evaluation datasets are too small and too clean to surface it.

In production, tool responses from real SaaS APIs are noisy. A Salesforce opportunity record returned via API contains dozens of fields, many of them null, many of them containing values that only make sense to a user who knows the internal naming conventions. A Jira ticket description is often a fragment written by an engineer in a hurry. When an agent receives that raw payload as context, it has to do significant interpretive work, and it will make errors that are invisible until a human reviews the output and notices something is wrong.

The engineering response to this is context enrichment at the gateway layer. Rather than passing raw API responses directly to the agent, the gateway post-processes tool outputs: it resolves IDs to human-readable labels, strips fields that carry no semantic value, joins related records where the agent would otherwise have to make a second tool call, and applies domain-specific normalization. This is unglamorous work. It is also where a significant fraction of agent quality improvement actually comes from.

Access Control Patterns for SaaS Integrations

OAuth Delegation vs. Service Accounts

The most common shortcut teams take is provisioning a single service account per SaaS integration and using that account's credentials for all agent tool calls. This is operationally simple and it breaks the moment you need to audit what an agent did on behalf of a specific user, or when the service account has broader permissions than any individual user should be delegating.

The correct pattern is OAuth 2.0 delegation with token exchange at the gateway boundary. The user authenticates to the agent platform, the platform exchanges that session for a scoped token in each connected SaaS system, and tool calls execute under that token. The agent's effective permissions in any given system are bounded by what the human principal can actually do in that system.

Scope Minimization at Registration Time

Each tool registered in the gateway should declare its required OAuth scopes explicitly, and the gateway should request only those scopes during the token exchange flow. This sounds obvious but most teams do not implement it because it requires maintaining a scope manifest per tool, which adds overhead during development. The operational benefit is that a compromised agent session has a bounded blast radius.

The Iceberg Problems That Only Surface at Scale

The surface-visible problems in MCP gateway work are the ones teams plan for: authentication, rate limiting, error handling. The iceberg problems are the ones that appear after the system has been running in production for a few weeks.

The first is schema drift. SaaS vendors update their APIs, field names change, response structures evolve, and the tool definitions registered in your gateway become stale. An agent that was producing correct outputs starts producing subtly wrong ones because a field it was relying on has been renamed or moved. Detecting this requires active schema validation on every tool response, not just at integration time.

The second is latency compounding. An agent that makes six sequential tool calls, each taking 400 milliseconds, takes 2.4 seconds before it can begin reasoning. In a multi-agent workflow where one agent's output is another agent's input, these delays stack. Gateway-level caching with appropriate TTLs for each data type is the mitigation, but deciding what to cache and for how long requires understanding the freshness requirements of each downstream use case.

What This Means for Your Scoping Process

Engineering leaders scoping this work for the first time consistently underestimate the ratio of gateway engineering to model engineering. In the Sierra build, the model integration was a relatively small fraction of total engineering effort. The majority was spent on the access control layer, context enrichment pipelines, schema validation, and the operational tooling needed to monitor and debug agent behavior in production.

That ratio is not a failure of planning. It reflects what the work actually is. Building an internal MCP gateway is fundamentally a platform engineering problem with an AI component, not an AI problem with a platform component. Teams that staff it as the latter will find themselves rebuilding foundational infrastructure under time pressure once production realities become apparent. The more productive framing is to treat the gateway as a first-class internal platform with its own roadmap, its own SLOs, and dedicated ownership, and to let the agent capabilities that sit on top of it evolve incrementally as that foundation stabilises.