Remote MCP servers need an operating model

A remote MCP server is an operational boundary, not a local server with a public URL taped onto it. The moment an agent can reach tools across a network, you own authentication, tenancy, request limits, deployment compatibility, logs, and the consequences of a model calling the wrong tool with valid credentials.

The transport choice follows that boundary. Use stdio when one user and one host own the process. Use Streamable HTTP when several clients, machines, or users need a shared service. Tunnels help during development, but they do not turn a laptop process into production. Serverless can work for short stateless calls, provided cold starts fit the client timeout and every downstream operation is safe to retry.

The Model Context Protocol changed materially in the 2026-07-28 specification. Current Streamable HTTP is stateless at the protocol layer, while deployed clients and servers may still speak older, session-based revisions. A hosting plan that ignores the negotiated protocol version will fail in ways that look like random routing, expired sessions, or broken streams. Decide which revisions you support before choosing infrastructure.

The ownership boundary chooses the transport

Choose stdio or Streamable HTTP by asking who starts the process, who may call it, and where its credentials live. Latency and hosting price matter later. A transport that contradicts the ownership model creates permanent security and support work.

With stdio, the MCP client launches a child process and exchanges newline-delimited JSON-RPC over standard input and output. The operating system already gives you a process boundary, a user identity, a working directory, and local credential sources. There is no listening port, public certificate, reverse proxy, or network authentication flow. That is an excellent fit for a developer tool, a desktop assistant, or an internal automation that runs beside its data.

Streamable HTTP makes the server an independently operated service. Clients send JSON-RPC requests to an HTTP endpoint, and the service can sit behind a gateway, load balancer, or private network. That is the right shape when a team needs one controlled implementation, when a SaaS product exposes user-specific tools, or when clients cannot install the server locally. It also means the server must identify the caller and enforce permissions on every operation.

Do not choose HTTP merely because a platform team knows how to deploy containers. A local filesystem tool exposed through HTTP now needs a remote identity model and a safe way to select files. Do not choose stdio when twenty employees need identical policy and audit records. Shipping and updating twenty local processes may cost more than operating one service.

A useful decision test has four questions:

• Does the client own the machine and start the server? Prefer stdio.
• Must two or more machines share the same server deployment? Prefer HTTP.
• Does the tool touch credentials that should never leave a user's machine? Keep it local unless you can redesign the credential flow.
• Must administrators revoke access centrally and review calls by identity? HTTP gives you the right control point.

Hybrid designs are legitimate. A small stdio adapter can keep local credentials and forward narrowly scoped requests to a remote service. That adapter should add a real boundary, such as local signing or data filtering. A wrapper that forwards every tool and argument unchanged only adds another component to debug.

Stdio is safer locally, not automatically safe

Stdio removes the network attack surface, but the spawned process still receives the client's authority. It can read environment variables, inherit filesystem access, launch child processes, and call the network under the user's identity. Treat installing a stdio server like installing a command-line program that an agent can operate.

The MCP transport specification is unusually strict about the two output streams. A server reads valid JSON-RPC messages from stdin, writes only valid protocol messages to stdout, and sends diagnostics to stderr. One innocent debug print on stdout can corrupt the framing. This failure often appears only in packaged builds because a development logger and a production logger use different defaults.

Use a launch wrapper that passes an allowlist of environment variables instead of the entire shell environment. Set an explicit working directory. Run the process as a user that cannot read unrelated repositories or SSH material. If the server invokes tools such as a database client or version control binary, pin the executable path rather than trusting PATH. These controls prevent a prompt-controlled argument from finding an unexpected executable earlier in the search path.

A minimal launch configuration should make authority visible:

{
  "command": "/opt/mcp/bin/inventory-server",
  "args": ["/work/inventory", "read_only"],
  "env": {
    "INVENTORY_API_URL": "https://inventory.internal",
    "LOG_LEVEL": "info"
  }
}

This fragment does not include a long-lived API token. Prefer a local credential helper or a short-lived credential source when the server supports one. If a token must enter the environment, scope it to the smallest API and never print the environment during error handling.

Restart behavior is another stdio advantage. The client can kill a wedged child and launch a clean process without coordinating with other users. It is also a trap if the server stores important state only in memory. Assume that the process can disappear between calls. Persist durable work in the system that owns it, and make tool results carry stable identifiers rather than references to memory objects.

Streamable HTTP is a service contract

Streamable HTTP earns its operational cost when remote MCP servers need shared access, central policy, or independent releases. It uses ordinary HTTP infrastructure, but the payload still carries JSON-RPC semantics, tool permissions, and version-specific MCP behavior. A generic reverse proxy will not supply those semantics for you.

In the current 2026-07-28 protocol, each request is self-contained. The old initialize and initialized exchange is gone, as is the protocol-level Mcp-Session-Id. Requests carry MCP-Protocol-Version, while Mcp-Method and, for named operations, Mcp-Name expose routing information to gateways. The official release notes argue that any request can reach any instance behind round-robin balancing. I agree for protocol state. Application state still needs an explicit handle, database record, queue job, or other shared representation.

That distinction matters. Stateless transport does not make a tool operation stateless. A report generator may start work, return a task handle, and finish later. A database transaction may depend on a user identity and an idempotency value. Put those facts in authenticated request context and durable storage. Do not hide them in one container's memory and then blame the load balancer when a later call reaches another instance.

Older revisions complicate the picture. Streamable HTTP in 2025-era specifications could issue Mcp-Session-Id, hold Server-Sent Events streams, and require session affinity or a shared session store. The older HTTP+SSE transport used separate connection behavior and is now deprecated. If you support these clients, route by negotiated version and test their full lifecycle. A server that accepts initialization, loses the session on scale-out, and returns 404 on the next tool call is technically explaining its state loss, but users will see an unreliable product.

Start with plain request and response behavior unless your supported revision and tool genuinely need streaming or subscriptions. Open streams consume proxy connections, complicate deploy draining, and meet idle timeouts in several layers. Streaming can improve perceived latency for long output, but it does not shorten the operation. A job handle plus polling is often easier to recover after a mobile connection changes networks.

Your health endpoint should prove process readiness without calling an MCP tool or a paid dependency. Put deeper checks in monitoring. If a load balancer exercises a real tool every few seconds, it can mutate data, consume quotas, or keep an otherwise idle server permanently warm.

Hosting options trade control for operational work

The best host is the least elaborate option that meets the server's latency, isolation, scaling, and network requirements. Remote MCP traffic is usually uneven: long quiet periods, then a burst of tool discovery and calls when an agent starts work. Average CPU tells you little about that pattern.

A small virtual machine or long-running container is the boring default. It starts once, keeps connection pools warm, supports older streaming clients, and gives predictable latency. You must patch the image, restart failed processes, configure certificates or a proxy, and plan capacity. For one internal server with modest traffic, that may still be less work than adapting every tool to serverless limits.

A managed container service removes host maintenance and can scale replicas. It suits current stateless HTTP well. Set a minimum instance count when interactive latency matters, and make shutdown graceful enough to finish or safely abort in-flight calls. Check whether the platform buffers streaming responses, enforces a request duration, or closes idle connections. Those limits often matter more than the advertised CPU and memory sizes.

Functions and scale-to-zero containers fit short, independent calls. They are a poor default for tools that hold subprocesses, maintain local indexes, depend on large SDK initialization, or stream for minutes. A function can also multiply downstream connections during a burst. Ten fresh instances opening database pools at once may overload a database that handled the same request rate from two warm containers. Limit concurrency and pool size with the downstream service in mind.

A Kubernetes cluster makes sense when the company already operates one and needs its network policy, workload identity, or placement controls. MCP alone does not justify a cluster. The protocol does not become safer because the pod has a YAML file. Someone still has to define ingress authentication, egress limits, secrets, disruption behavior, and per-tool authorization.

Edge runtimes are attractive for global TLS termination and low connection latency. Verify runtime support for the server SDK, streaming, cryptography, outbound sockets, and response duration. If every tool immediately calls a regional database, moving only the MCP endpoint to the edge can add a second network leg without reducing completion time. Place execution near the slow or sensitive dependency, then use a global gateway only where it helps.

Compare options with measurements from one representative tool call, not a hello-world endpoint. A long-running VM fits a small internal service with stable load, but you must test process restarts during calls. A managed container fits a shared stateless service, but platform limits, minimum instances, and scale-out pressure on downstream quotas need direct tests. A serverless function fits short independent operations, provided retries after timeout cannot duplicate work. Existing Kubernetes fits workloads that already depend on cluster policy, but drain, reschedule, and network policy deserve failure tests. An edge runtime fits lightweight calls near distributed users only when its restrictions and regional backend path do not terminate or delay the work.

Tunnels are for discovery, not trust

A tunnel is the fastest way to let a remote client reach a server on a developer machine. It is useful for callback testing, mobile clients, client compatibility, and showing a teammate a work in progress. The tunnel supplies reachability and usually TLS. It does not supply a production authorization design, stable capacity, or a durable operator.

The dangerous pattern is an unauthenticated local server bound to 0.0.0.0, followed by a tunnel command that creates a public URL. That server may have inherited cloud credentials and repository access. A random URL is not a secret, and tunnel access logs do not compensate for missing application authorization. Put authentication at the MCP service, even for a short test, and scope the test credential to disposable data.

Bind the development server to 127.0.0.1, then point the tunnel at that listener. The MCP transport specification tells local HTTP servers to bind to localhost and validate the Origin header to reduce DNS rebinding risk. Origin validation is not a replacement for authentication. Non-browser clients may omit Origin, while an allowed web origin says nothing about which employee or tenant made a tool call.

Treat each tunnel URL as temporary. Remove it from client configuration after the test, revoke associated tokens, and confirm the tunnel process has stopped. Do not publish the URL in a shared example that survives for months. If a demonstration must stay online, deploy a small managed instance with an owner, budget, alerts, and an expiration date.

Tunnels also distort performance tests. The connection may traverse a relay region, the development laptop may sleep, and local dependency caches may be warmer than a clean deployment. Use tunnels to test protocol compatibility and user flow. Use the intended hosting region and instance shape to measure latency, cold starts, concurrency, and failure recovery.

Cold starts consume the client's timeout budget

Cold starts are acceptable only when the entire path still completes inside the client's deadline with room for retries and network variance. Measure process start, runtime initialization, SDK imports, secret retrieval, connection setup, tool execution, and response serialization separately. The platform's startup chart covers only part of that path.

Interactive MCP clients may display a generic tool failure when their own timeout expires. Meanwhile, the server may continue running and commit the operation. The client retries, and a non-idempotent tool creates a duplicate issue, charge, deployment, or message. This is the cold-start failure that deserves attention: uncertainty about whether work happened, not merely a slow first token.

Design mutating tools around idempotency. Accept or derive an operation identifier, store it before side effects, and return the earlier result when the same authenticated caller repeats it. Keep the identifier scoped to the caller and operation so one tenant cannot probe another tenant's result. For a long task, return a durable handle quickly and provide a status operation rather than holding one HTTP request until every dependency finishes.

Warm-up requests can hide symptoms but create their own cost and false confidence. A health ping may warm the runtime without loading the same libraries, credentials, or database path as a tool call. Minimum instances give more predictable behavior, but they turn scale-to-zero pricing into a small always-on service. Compare that price with a managed container before building a warming system.

Collect a latency breakdown by tool name and instance state. Record queue delay, cold or warm status, downstream duration, and final outcome. Do not log full arguments by default because prompts and tool parameters often contain customer data. A trace ID and sanitized dimensions are enough to connect gateway, server, and dependency records.

Set timeouts in descending order from the client inward. The client should wait longer than the gateway, the gateway longer than the application deadline, and the application deadline longer than its downstream timeout. That order gives the server a chance to return a precise error before an outer layer cuts the connection. It also leaves time to record whether a side effect completed.

Security posture follows tool authority

A private network lowers exposure but does not establish caller identity. Workload identity at the gateway can identify the client service, while user-delegated identity tells the handler whose data and permissions apply. Many internal deployments have only the first. Every employee then acts through one shared service account, so revoking one person or explaining one destructive call becomes impossible. Carry both identities when a service calls on behalf of a user, and reject a request when the delegation chain is missing.

Separate control-plane tools from ordinary data tools. A tool that changes server configuration, rotates credentials, or grants access should not share the same scope as a tool that reads a ticket. If the client discovers both in one catalog, a prompt injection in ordinary content can aim at the administrative action. Put privileged tools behind stronger authorization and expose them only to clients that need them. Hiding a tool description from the model helps reduce accidents, but server policy must enforce the boundary.

Outbound access needs the same attention as inbound access. A search or fetch tool that accepts an arbitrary URL can become a path to cloud metadata, internal dashboards, and services that trust the server network. Resolve and validate destinations, block private and link-local ranges unless explicitly required, recheck after redirects, and limit response size. Run high-risk connectors with an egress policy that permits named dependencies instead of the whole internet.

Secrets should arrive through workload identity or a managed secret source, not baked into an image or copied into client configuration. Rotate them without rebuilding the server, and test rotation while instances are running. A secret that changes successfully in storage but remains cached for days in warm workers has not been operationally rotated.

Secure the action the tool can perform, not merely the endpoint that lists it. A server that authenticates a user and then lets every user call delete_project has authentication but no useful authorization. Map identity, tenant, scope, and resource ownership at the tool handler before touching a downstream system.

For remote HTTP, use TLS and short-lived bearer tokens obtained through the MCP authorization model or an equivalent organization-controlled mechanism. The current specification builds on OAuth conventions and protected resource metadata. Bearer tokens deserve the warning in RFC 6750: possession is enough to use them. Never place them in URLs, tool arguments, or logs. Validate issuer, audience, expiry, signature, and scopes at the service that relies on them.

The MCP server must not pass a client's token through to an unrelated upstream API. Token passthrough confuses the intended audience and prevents the server from applying its own service identity. Exchange or obtain a token meant for the downstream resource, then enforce that the caller may request the action. This keeps the trust chain inspectable.

Validate Origin on Streamable HTTP connections, reject unexpected hosts, limit body and header sizes, and apply request rates by authenticated identity. In the 2026-07-28 protocol, gateways can inspect Mcp-Method and Mcp-Name, but the server must reject a mismatch between those headers and the JSON-RPC body. Otherwise a gateway could authorize a harmless listed method while the body asks the server to execute a different one.

Tool schemas are input validation, not authorization. Validate types, lengths, enumerations, and path boundaries again in the handler. Resolve filesystem paths before checking that they stay under an allowed root. Parameterize database queries. Restrict outbound destinations so a URL-fetching tool cannot reach instance metadata, control planes, or private admin services.

Use separate credentials and, when practical, separate deployments for read-only and destructive tools. Human approval in the client helps, but the server cannot assume every client displays approval correctly. High-impact actions should require a stronger scope, a server-side policy check, or a two-stage operation that previews a concrete change and then commits that exact version.

Logs need an actor, tenant, tool name, request ID, policy decision, duration, and result class. Keep sensitive arguments out unless an investigation setting explicitly captures them with access controls and retention. Audit records should tell you who authorized an action and what resource changed without becoming a second copy of every document the agent touched.

Compatibility must be tested as a matrix

A remote MCP server can pass unit tests and still fail because the client, proxy, and server disagree about protocol revision or streaming behavior. Publish the revisions and transports you support, then test those exact combinations. Compatibility by accident disappears during the next SDK update.

Include a current stateless client, the oldest supported session-based client, and any legacy HTTP+SSE client you have promised to keep. Run each through the same ingress used in production. Direct localhost tests will not reveal proxy buffering, header removal, idle connection limits, or path rewriting.

A useful release gate exercises these cases:

1. Discover or initialize according to the client's negotiated revision, then list tools and call one read-only tool.
2. Start a long response, restart one server instance, and confirm the client receives a defined failure or resumes only where the revision permits it.
3. Send mismatched Mcp-Method or Mcp-Name headers and verify rejection before execution.
4. Expire or revoke credentials, then confirm both existing and new requests lose access as designed.
5. Repeat one mutating call with the same operation identifier and verify a single side effect.

Capture the status code, JSON-RPC error, and server record for each failure. A proxy-generated 502, an application 401, and a JSON-RPC tool error require different remedies. If monitoring collapses all three into tool_failed, the on-call engineer will waste the first half hour reproducing what the telemetry should already show.

Version routing deserves restraint. Prefer one endpoint that correctly negotiates supported revisions. If old and current implementations need different state models, split them behind an explicit compatibility layer and measure old-client use. Do not keep sticky sessions, a session database, and an SSE fleet forever for clients nobody can name. Set a retirement criterion and communicate it.

A production design should be boring to operate

A sound default for new remote MCP servers is a stateless Streamable HTTP service on a managed container platform, behind an authenticating gateway, with at least one warm instance when humans wait on calls. Keep application state in a durable system, make mutations idempotent, issue short-lived credentials, and restrict each tool to the smallest downstream authority. This design scales without relying on invisible process memory.

Put four ownership lines in the service record: who owns tool behavior, who owns identity policy, who responds to availability alerts, and who approves destructive scope. Small teams may assign all four to one person, but writing them down prevents an MCP experiment from becoming anonymous infrastructure. Record the supported protocol revisions beside those names.

Before launch, budget for the failure path. Decide what happens when the model sends invalid arguments, the user's token expires, a downstream API returns slowly, an instance stops mid-call, or the client retries. A clean error should say whether the operation is safe to retry. For a mutation, the status operation should resolve uncertainty without repeating the action.

In a Team & AI Audit, I treat MCP hosting as part of the engineering operating model, because a clever tool that only one developer can deploy or secure does not reduce team cost. The useful question is whether the service removes repeated work while leaving a support path that one or two engineers can actually own.

Do not promote the tunnel. Do not add Kubernetes to make the diagram look serious. Run the compatibility matrix, inspect the authority of every tool, and force one instance to die during a call. The hosting choice is ready when that failure produces a defined result instead of a mystery.