What is MCP?

MCP (Model Context Protocol) is a standardized protocol for exposing tools, resources, and prompts to AI applications/agents.

Agent / AI Application
        │
        │ MCP
        ▼
   MCP Server
        │
   ┌────┼────┐
   ▼    ▼    ▼
 REST  SDK  DB
 API

The MCP server doesn’t necessarily implement the business capability itself. It can wrap existing systems, APIs, SDKs, databases, etc.

The key question

Don’t ask: "Do I have many tools? Should I create MCP?"

Ask: "Do I need to expose these capabilities as a reusable, standardized AI interface?"

That’s the decision point.

When you SHOULD create an MCP Server

1. Multiple agents need the same capabilities

For example:

Finance Agent ─────┐
Procurement Agent ─┤
SRE Agent ─────────┤
Security Agent ────┤
                   ▼
             ServiceNow
              MCP Server

Tools:

get_incident()
create_incident()
update_incident()
search_incidents()
get_change_request()

Instead of implementing those integrations independently in every agent, create an MCP server.

2. Multiple AI applications/frameworks need the capability

For example:

OpenAI Agent
LangGraph Agent
Google ADK Agent
Claude Agent
Custom Enterprise Agent
        │
        ▼
    MCP Server
        │
        ▼
     SAP

MCP gives them a common interface.

3. You want capability discovery

An AI client can discover what the MCP server exposes.

Conceptually:

Client
   │
   │ "What can you do?"
   ▼
MCP Server
   │
   ├── search_customer
   ├── get_order
   ├── create_order
   └── cancel_order

This is much more useful in dynamic agent ecosystems.

4. You want a reusable AI capability boundary

Suppose your organization has:

SAP
ServiceNow
Salesforce
AWS
GitHub
Jira

Rather than every AI team building their own integration:

Agent A → SAP
Agent B → SAP
Agent C → SAP
Agent D → SAP

you can provide:

                 SAP MCP Server
                       │
          ┌────────────┼────────────┐
          ▼            ▼            ▼
       Agent A       Agent B      Agent C

That becomes an enterprise AI capability.

5. You want centralized governance

An MCP server can become a controlled boundary for:

• authentication
• authorization
• tool permissions
• audit
• logging
• rate limiting
• secrets
• policy enforcement
• tool versioning

For enterprise environments, this can be a significant reason.

When you SHOULD NOT create an MCP Server

1. Tool is only for one agent

Example:

Invoice Agent
     │
     ├── extract_invoice()
     ├── validate_invoice()
     ├── calculate_tax()
     └── generate_report()

If these capabilities are:

• private
• tightly coupled
• only needed by this agent
• unlikely to be reused

just use normal tools.

Agent
 │
 ├── Tool 1
 ├── Tool 2
 ├── Tool 3
 └── Tool 4

No MCP required.

2. Even 100 tools can still be normal tools

This is important.

You might have:

Agent
 │
 ├── Tool 1
 ├── Tool 2
 ├── ...
 └── Tool 100

If all 100 are specific to one application/use case, MCP doesn’t automatically provide value.

Tool count is not the deciding factor.

3. The capability is tightly coupled to the agent

Suppose: Fraud Detection Agent

has internal functions such as:

calculate_fraud_score()
build_fraud_features()
apply_fraud_model()

Those are implementation details of the agent. Turning them into an MCP server could actually make the architecture unnecessarily complicated.

4. There is no meaningful reuse

If the architecture is:

Agent A
   │
   ▼
MCP Server
   │
   ▼
One private function

and nobody else will consume it, ask: What problem did MCP solve?

If the answer is essentially “nothing,” don’t introduce it.

The simplest decision matrix

The architecture

Private capabilities

                 Agent
                   │
          ┌────────┼────────┐
          ▼        ▼        ▼
       Tool A    Tool B   Tool C

Keep them internal.

Shared capabilities

                 Agent A
                    │
                 Agent B
                    │
                 Agent C
                    │
                    ▼
               MCP Client
                    │
                   MCP
                    │
                    ▼
              MCP Server
                    │
          ┌─────────┼─────────┐
          ▼         ▼         ▼
        REST       SDK       DB
          │         │         │
          ▼         ▼         ▼
       Enterprise Systems

One important distinction

MCP does not mean “better tool.”

It means: “I am exposing my capabilities through a standardized protocol so AI clients can consume them.”

So:

Tool
=
Capability available to an agent

MCP Server
=
Standardized server exposing capabilities to AI clients

And:

REST API
=
General service-to-service interface

SDK
=
Developer/programming interface

CLI
=
Command-line interface

MCP
=
AI/agent-facing protocol

The rule

• Use native/internal tools for capabilities that are private to an agent or a specific application.

• Create an MCP server when capabilities need to become reusable, discoverable, standardized, or governed across multiple agents, AI applications, teams, or systems.

• Do not introduce MCP merely because the number of tools is large.

What is the Model Context Protocol (MCP)?

MCP (Model Context Protocol) is an open-source standard for connecting AI applications to external systems.

Think of MCP like a USB-C port for AI applications. Just as USB-C provides a standardized way to connect electronic devices, MCP provides a standardized way to connect AI applications to external systems.

Why does MCP matter?

Depending on where you sit in the ecosystem, MCP can have a range of benefits.

• Developers: MCP reduces development time and complexity when building, or integrating with, an AI application or agent.
• AI applications or agents: MCP gives them access to an ecosystem of data sources, tools and apps, which enhances their capabilities and improves the end-user experience.

Architecture overview

Scope

• MCP Specification: A specification of MCP that outlines the implementation requirements for clients and servers.
• MCP SDKs: SDKs for different programming languages that implement MCP.
• MCP Development Tools: Tools for developing MCP servers and clients, including the MCP Inspector
• MCP Reference Server Implementations: Reference implementations of MCP servers.

Concepts of MCP

MCP follows a client-server architecture where an MCP host — an AI application establishes connections to one or more MCP servers.The MCP host accomplishes this by creating one MCP client for each MCP server. Each MCP client maintains a dedicated connection with its corresponding MCP server.

• Local MCP servers that use the STDIO transport typically serve a single MCP client

• Remote MCP servers that use the Streamable HTTP transport will typically serve many MCP clients.

• MCP Host: The AI application that coordinates and manages one or multiple MCP clients

• MCP Client: A component that maintains a connection to an MCP server and obtains context from an MCP server for the MCP host to use.

• MCP Server: A program that provides context to MCP clients.

Example:

• Visual Studio Code acts as an MCP host.
• When Visual Studio Code establishes a connection to an MCP server, such as the Sentry MCP server, the Visual Studio Code runtime instantiates an MCP client object that maintains the connection to the Sentry MCP server.
• When Visual Studio Code subsequently connects to another MCP server, such as the local filesystem server, the Visual Studio Code runtime instantiates an additional MCP client object to maintain this connection.

Note: that MCP server refers to the program that serves context data, regardless of where it runs.

• MCP servers can execute locally or remotely.

Layers

MCP consists of two layers:

• Data layer: Defines the JSON-RPC based protocol for client-server communication, including capability and version discovery, and core primitives, such as tools, resources, prompts and notifications.
• Transport layer: Defines the communication mechanisms and channels that enable data exchange between clients and servers, including transport-specific connection establishment, message framing, and authorization.

Data layer = what the MCP messages mean. Transport layer = how those messages travel between the MCP client and server.

And a local MCP server can still use both layers.

MCP architecture

Think of MCP like HTTP + JSON-RPC, but with MCP-specific semantics:

┌─────────────────────────────────────────────┐
│                 MCP CLIENT                  │
│                                             │
│  "I want to call the Sentry search tool"    │
└──────────────────────┬──────────────────────┘
                       │
                       │ Transport Layer
                       │
                       ▼
              ┌─────────────────┐
              │  MCP Messages   │
              │                 │
              │ JSON-RPC        │
              └────────┬────────┘
                       │
                       │ Data Layer
                       ▼
┌─────────────────────────────────────────────┐
│                MCP SERVER                   │
│                                             │
│ Tools / Resources / Prompts / Notifications │
└─────────────────────────────────────────────┘

More precisely, JSON-RPC belongs to the MCP data layer, while the transport determines how those JSON-RPC messages are carried.

Data Layer — WHAT is being communicated?

The data layer defines the meaning and structure of MCP communication.

For example, the client can say: What tools do you provide?

The MCP message might conceptually look like:

{
  "jsonrpc": "2.0",
  "id": 1,
  "method": "tools/list"
}

The server might respond:

{
  "jsonrpc": "2.0",
  "id": 1,
  "result": {
    "tools": [
      {
        "name": "search_issues",
        "description": "Search Sentry issues"
      }
    ]
  }
}

That’s data-layer communication.

It defines things such as:

• Initialization: initialize
• Tools: tools/list, tools/call
• Resources: resources/list, resources/read
• Prompts: prompts/list, prompts/get
• Notifications: notifications/...

It also handles things such as:

• Protocol version negotiation
• Capability discovery
• Requests
• Responses
• Errors
• Notifications

So the data layer answers: What does this MCP message mean?

Transport Layer — HOW does the message travel?

Now imagine you have this JSON:

{
  "jsonrpc": "2.0",
  "id": 1,
  "method": "tools/list"
}

The transport layer determines how that message gets from the client to the server.

Example:

Client
  │
  │ JSON-RPC message
  │
  ▼
Transport
  │
  ▼
Server

MCP supports different transport mechanisms.

The important ones to understand are:

STDIO

Client
   │
   │ stdin/stdout
   ▼
MCP Server Process

Streamable HTTP

Client
   │
   │ HTTP
   ▼
MCP Server

So the same conceptual MCP operation: tools/list can travel over different transports.

This is the key distinction

Suppose you have: tools/call. That’s data layer. How does it travel?

Option A

tools/call
     ↓
STDIO
     ↓
MCP Server

Option B

tools/call
     ↓
HTTP
     ↓
MCP Server

The MCP operation doesn’t fundamentally change. Only the transport changes.

MCP Client Best Practices

Progressive Tool Discovery

• Naive MCP host implementations pass the tool definitions of every connected server directly to the model at the start of each conversation.For a handful of tools, this is perfectly reasonable.

• But when a host has access to dozens of servers exposing hundreds of tools, those definitions alone can consume the majority of the context window before the model has even read the user’s message.

Progressive discovery avoids this:

• The host fetches tool definitions via tools/list as normal, but defers injecting them into the model’s context.

• The host provides a lightweight search_tools meta-tool to the model.

• The host loads full definitions into context only as needed.

When to Use Progressive Discovery

Progressive discovery is best used when tool definitions take large parts of the context window. For a small set of tools with tool definitions taking up a small part of the context window, loading all tools is fine. Once the tool definitions take up a significant part of the available context window, clients should switch to progressive discovery. We recommend that clients implement thresholds to determine when to switch:

• Implement a threshold as a percentage of the context window. For example, 1%-5%.
• Load tool definitions. Once the threshold is reached, switch to progressive discovery.

Choosing a Discovery Strategy

Once the model invokes the search_tools tool, we need to choose a search strategy:

• Keyword-based: Keyword matching (BM25, regex). Simple and effective, particularly for descriptive tool names and descriptions.
• Embedding-based: Vector-similarity retrieval over tool descriptions. Handles synonyms and semantic matching better.
• Subagent-based: A secondary model, often a small and fast model such as Claude Haiku or Gemini Flash, selects tools for the task. This usually works very well but can be more costly than embedding-based or keyword-based solutions.
• Hybrid: Combine approaches. For example, by scoring across keyword and embedding rankings, or choosing different strategies depending on use-case or query.

Some model providers already offer built-in tool search. For example, OpenAI and Anthropic support this natively; check your provider’s documentation for an equivalent. When available, you may prefer the platform’s tool search over a custom implementation. Build your own when the provider doesn’t offer one or when you need specialized retrieval logic (e.g., domain-specific ranking or access-control filtering).

The three-layer pattern below illustrates a custom search-based approach in detail, but the layered principle (catalog, inspect, execute) applies regardless of retrieval mechanism.

Using Progressive Discovery

One common implementation for progressive discovery uses a search-based three-layer approach:

Layer 1: Catalog. The host exposes a small set of meta-tools for searching available capabilities. A search_tools tool accepts a natural-language query and returns matching tool names with brief descriptions.

// The model calls a lightweight search tool
search_tools({ query: "update salesforce record" })

// Returns concise matches: names and one-line descriptions only
→ [
    { name: "salesforce_updateRecord", description: "Update fields on a Salesforce object" },
    { name: "salesforce_upsertRecord", description: "Insert or update based on external ID" }
  ]

Layer 2: Inspect. Once the model identifies a candidate, it fetches the full definition (input schema, output schema, documentation) for that tool only.

// The model inspects only the tool it needs
get_tool_details({ name: "salesforce_updateRecord" });

Layer 3: Execute. The model calls the tool with full knowledge of its interface, having loaded only the definitions it needed.

This pattern reduces token usage dramatically and can improve tool selection accuracy: the model focuses on a few relevant tools rather than scanning hundreds of irrelevant ones. Other discovery strategies (embeddings, subagents, etc.) follow the same layered principle but substitute different retrieval mechanisms in the catalog layer.

Dynamic Server Management

Progressive discovery extends beyond individual tools to entire servers. Rather than connecting to every configured server at startup, a host can:

1. Maintain a registry of available servers and their high-level descriptions.
2. Connect to a server only when the model determines it needs that server’s capabilities.
3. Disconnect servers that are no longer relevant to the current task, freeing context.

This works especially well for general-purpose agents, where the user’s intent isn’t known upfront. The agent starts with a minimal set of always-on servers and connects others as needed. Combined with agent skills, a skill file can declare which MCP servers it needs, and the host connects them only when that skill is invoked.

Implementation Guidelines

When implementing progressive discovery:

Caching

MCP Caching — Summary

1. What is MCP Caching?

MCP clients can cache information received from MCP servers instead of requesting the same information repeatedly.

Common cached results include:

• tools/list
• server/discover
• resources/read

This reduces unnecessary requests and improves performance.

---

2. ttlMs — Time To Live

ttlMs tells the MCP client how long a cached result can be considered fresh.

Example:

{
  "ttlMs": 300000
}

300000 milliseconds = 5 minutes.

Conceptually:

tools/list
    ↓
Cache result
    ↓
TTL = 5 minutes
    ↓
After 5 minutes → Cache becomes stale
    ↓
Fetch fresh result

Key point

ttlMs = time-based cache expiration.

---

3. cacheScope

cacheScope provides information about the scope in which the cached result can be reused.

It should be interpreted according to the MCP caching specification.

Do not confuse:

ttlMs       → How long?
cacheScope  → Where/how broadly can it be reused?

---

4. list_changed Notification

An MCP server can notify the client when its available items have changed.

For tools:

notifications/tools/list_changed

Example:

Initial tools:

Search
Create
Delete

The client caches this list.

Later, the server adds:

Update

The server sends:

notifications/tools/list_changed

The client should immediately consider the cached tool list stale.

---

5. TTL vs list_changed

There are two cache invalidation mechanisms:

Important rule

list_changed takes precedence over the remaining TTL.

Example:

TTL = 1 hour

10:00 → tools/list
       ↓
      Cache

10:15 → Server changes tools
       ↓
       list_changed
       ↓
      Cache is stale NOW

The client does not wait until 11:00.

---

6. Host-Side Cache vs Model Context

These are different concepts.

                 MCP Client
                     │
          ┌──────────┴──────────┐
          │                     │
          ▼                     ▼
   Host-side cache         Model context
          │                     │
          │                     │
   Stores tool definitions   Tools currently
   for reuse                 provided to LLM

The client might cache:

1000 tool definitions

but only provide:

20 relevant tools

to the model.

Therefore:

Caching tool definitions does not mean all cached tools are placed into the model’s context.

---

7. Why Caching Matters for Large MCP Systems

An agent may connect to many MCP servers:

                     AI Agent
                        │
                    MCP Client
                        │
          ┌─────────────┼─────────────┐
          ▼             ▼             ▼
      GitHub MCP    Sentry MCP   ServiceNow MCP
          │             │             │
       50 tools       30 tools      100 tools

Without caching:

Repeated requests
       ↓
tools/list
tools/list
tools/list
...

With caching:

MCP Server
     ↓
tools/list
     ↓
MCP Client Cache
     ↓
Reuse until:
     ├── TTL expires
     └── list_changed arrives

---

8. Complete Flow

MCP Server
     │
     │ tools/list
     ▼
MCP Client
     │
     ▼
┌──────────────────────┐
│ Tool Definition Cache│
│                      │
│ ttlMs                │
│ cacheScope           │
└──────────┬───────────┘
           │
           │ Relevant tools
           ▼
      Model Context
           │
           ▼
           LLM

If the TTL expires:

Cache
  ↓
Stale
  ↓
tools/list
  ↓
Fresh data

If the server sends:

notifications/tools/list_changed

then:

Cache
  ↓
Immediately stale
  ↓
tools/list
  ↓
Fresh data

---

9. Architect-Level Mental Model

Remember these three concepts:

ttlMs

Time-based freshness

cacheScope

Scope of cache reuse

list_changed

Server-driven immediate invalidation

In short:

MCP caching allows clients to reuse server results efficiently. ttlMs controls time-based expiration, while list_changed allows the server to invalidate cached lists immediately when their contents change.

MCP Prompt Caching — Summary

1. What is Prompt Caching?

LLM providers can cache a stable prefix of the prompt so they don’t have to process the same content repeatedly.

A typical request may contain:

System Instructions
        +
Conversation
        +
Tools Array
        +
User Message
        ↓
       LLM

The tools array can be part of the cached prompt prefix.

---

2. Why MCP Tools Affect Prompt Caching

MCP clients may dynamically discover tools from MCP servers.

For example:

Initial tools:

[A]
[B]
[C]

Later a new tool is discovered:

[A]
[B]
[C]
[D]

Changing the tool definitions can affect the LLM’s prompt cache.

If the cache is invalidated, the provider may need to process the larger prompt again, potentially increasing token costs.

---

3. Keep the Tools Array Stable

When adding newly discovered tools, prefer appending them instead of re-sorting the entire array.

Preferred

Before:

[A]
[B]
[C]

After:

[A]
[B]
[C]
[D]

Avoid unnecessary reordering

Before:

[A]
[B]
[C]

After:

[D]
[A]
[B]
[C]

Keeping the existing prefix stable gives the provider a better opportunity to reuse the prompt cache.

---

4. MCP Tool Cache vs Prompt Cache

These are two different caching mechanisms.

Conceptually:

MCP Server
     │
     │ tools/list
     ▼
MCP Client
     │
     ▼
┌───────────────────┐
│ MCP Tool Cache    │
│                   │
│ Tool definitions  │
└─────────┬─────────┘
          │
          │ selected tools
          ▼
┌───────────────────┐
│ LLM Prompt        │
│                   │
│ System prompt     │
│ Conversation      │
│ Tools             │
└─────────┬─────────┘
          │
          ▼
    Prompt Cache
          │
          ▼
         LLM

Important

Caching MCP tool definitions does not mean those tools are currently in the model’s context.

The client can cache many tools but provide only the relevant tools to the model.

---

5. Progressive Tool Discovery

If an agent has many MCP servers and hundreds or thousands of tools, putting every tool into the LLM context is inefficient.

Instead, the client can use a search mechanism:

LLM
 │
 │ search_tools("Sentry errors")
 ▼
MCP Host
 │
 │ Search cached tool catalog
 ▼
Relevant tools
 │
 ▼
LLM

This reduces context usage.

---

6. Stable call_tool() Meta-Tool

Another approach is to expose one stable tool to the LLM:

call_tool({name, args})

Instead of exposing hundreds of changing tools:

search_github
create_issue
update_issue
delete_issue
...

the LLM always sees:

call_tool

Example:

{
  "name": "call_tool",
  "arguments": {
    "name": "create_issue",
    "args": {
      "title": "Login failure"
    }
  }
}

The host then routes the call to the appropriate MCP server.

LLM
 │
 │ call_tool()
 ▼
MCP Host
 │
 ├── GitHub MCP
 ├── Sentry MCP
 └── ServiceNow MCP

Benefit

The LLM’s tool interface remains stable, which can help preserve prompt caching.

---

7. Server Disconnection

Disconnecting an MCP server can change the available tool set.

For example:

Conversation
+
GitHub tools
+
Sentry tools

After disconnecting Sentry:

Conversation
+
GitHub tools

The tool definitions have changed, potentially affecting prompt caching.

Therefore, the guidance is to treat server disconnection as a conversation-boundary operation, rather than repeatedly connecting/disconnecting servers on every turn.

---

8. Three Important Optimization Layers

              MCP System
                  │
       ┌──────────┼──────────┐
       ▼          ▼          ▼
 MCP Tool Cache  Progressive  Prompt Cache
                 Discovery
       │          │          │
       ▼          ▼          ▼
 Avoid repeated  Reduce      Avoid repeated
 tools/list      context     LLM processing
 requests        size        of stable prefix

MCP Tool Cache

Avoid unnecessary MCP requests.

Progressive Tool Discovery

Avoid putting unnecessary tools into the model context.

Prompt Caching

Avoid repeatedly processing the same prompt prefix.

---

9. Architect-Level Mental Model

                    MCP SERVER
                        │
                        │ tools/list
                        ▼
                   MCP CLIENT
                        │
                        ▼
                MCP Tool Cache
                        │
                 Search/select
                        │
                        ▼
                Relevant Tools
                        │
                        ▼
                ┌──────────────┐
                │ LLM Request  │
                │              │
                │ System       │
                │ Conversation │
                │ Tools        │
                └──────┬───────┘
                       │
                       ▼
                 Prompt Cache
                       │
                       ▼
                      LLM

10. Key Takeaway

MCP tool caching saves MCP/network work. Progressive tool discovery saves model context. Prompt caching saves repeated LLM processing and can reduce token costs.

A well-designed MCP client should consider all three.

Programmatic Tool Calling / Code Mode — Summary

1. What is Programmatic Tool Calling?

Programmatic Tool Calling (Code Mode) allows an LLM to write code that invokes tools instead of making each tool call directly.

Direct Tool Calling

The flow is:

LLM → Tool → Result → LLM → Tool → Result → LLM

Every intermediate tool result is returned to the model and placed into its context.

This can cause:

• High token consumption
• Larger context windows
• Increased latency
• Unnecessary transfer of intermediate data through the model

Programmatic Tool Calling

The flow becomes:

LLM
 ↓
Generated Code
 ↓
Sandbox
 ↓
MCP Host / Broker
 ↓
MCP Servers

The sandbox executes the generated code, while intermediate tool results remain inside the sandbox.

Only the final result, such as a console.log() summary, is returned to the model.

---

2. How It Works

Programmatic tool calling has three main steps.

Step 1 — Generate a Programmatic API

The MCP Host reads MCP tool definitions and generates typed functions that the sandbox can call.

For example:

function logging_getLogs(input: {
  level: "error" | "warn" | "info";
  since: number;
}): Promise {
  return mcp.callTool(
    "logging_getLogs",
    input
  );
}

The generated API is based on the MCP tool schema and, when available, its outputSchema.

---

3. outputSchema

An MCP server can optionally provide an outputSchema for its tools.

When available, the host can generate precise return types.

Example:

interface LogEntry {
  timestamp: string;
  message: string;
  level: string;
}

If outputSchema is unavailable

The source recommends two approaches:

1. Use a generic type such as any or string.
2. For single-shot calls outside loops, use a fast model to extract a typed result.

The second approach introduces additional latency and may hallucinate or drop fields, so the result should be validated against the expected type.

---

4. Step 2 — Model Writes Code

Instead of asking the model to make multiple independent tool calls, the model generates one program that can chain calls together.

Example workflow:

Get error logs
      ↓
Filter errors
      ↓
Deduplicate errors
      ↓
Create ticket for each unique error
      ↓
Return summary

The filtering and deduplication happen inside the sandbox rather than inside the model’s context.

This prevents potentially thousands of log entries from being passed through the LLM.

---

5. Step 3 — Sandbox Executes the Code

The sandbox executes the model-generated program.

Function calls made by the code are intercepted and routed through the MCP Host.

Sandbox
   │
   ├── MCP Tool Call → Host → MCP Server A
   │                         ↓
   │                       Result
   │
   └── MCP Tool Call → Host → MCP Server B
                             ↓
                           Result

The intermediate data stays within the execution environment.

The model typically receives only:

console.log(...)

or a final return value.

---

6. Execution Architecture

The architecture consists of three major components.

Model

The model:

• Understands the task
• Determines which tools are required
• Generates code
• Receives the final summarized result

Sandbox

The sandbox:

• Executes model-generated code
• Provides typed tool functions
• Has no direct network access
• Keeps intermediate data out of the model context

Host / Broker

The host:

• Acts as the MCP host
• Receives function calls from the sandbox
• Routes calls to the correct MCP server
• Handles authentication and authorization
• Returns tool results to the sandbox

Credentials and API tokens remain with the host.

---

7. Sandbox Options

The source identifies several possible sandbox approaches:

These are examples rather than endorsements.

---

8. Important Integration Pattern

Regardless of the sandbox technology, the integration pattern remains the same:

Model-generated code
        ↓
Sandbox
        ↓
Function stubs
        ↓
Host Broker
        ↓
MCP Server
        ↓
Tool Result
        ↓
Sandbox
        ↓
Final Output
        ↓
LLM

The sandbox should communicate with the host through an in-process or stdio mechanism so that direct network access can remain disabled.

---

9. Security Considerations

Programmatic tool calling introduces a code execution surface, so sandboxing is critical.

Per-call Authorization

Approval of the generated script should not automatically approve every tool call it makes.

The broker must evaluate individual calls against the applicable authorization.

Cross-server Data Flow

Results from one MCP server should be treated as untrusted input when passed to another MCP server.

The broker should apply appropriate input-review policies.

Network Isolation

The sandbox should have:

No direct network access

External communication should go through the host broker.

Credential Protection

API keys and authentication tokens should remain with the host.

Generated code should only call typed functions.

Resource Limits

The sandbox should have limits such as:

• Execution timeouts
• Memory limits

This prevents runaway programs.

Output Filtering

Sandbox output should be:

• Validated
• Truncated where necessary
• Carefully filtered before being returned to the model

---

10. Error Handling

MCP tool errors can be returned with:

isError: true

rather than being treated as transport failures.

Generated wrappers should convert these errors into exceptions so model-generated code can use:

try {
  // MCP tool call
} catch (error) {
  // Handle error
}

If an uncaught error terminates the script, the error should be surfaced to the model so it can potentially self-correct.

The model should also report any partial side effects that have already occurred.

---

11. Programmatic Tool Calling vs Direct Tool Calling

---

12. Key Example

Consider the task:

Find all error logs from the past hour and create a ticket for each unique error.

Direct Tool Calling

Get logs
 ↓
Thousands of log entries
 ↓
LLM
 ↓
Analyze / deduplicate
 ↓
Create ticket
 ↓
LLM
 ↓
Repeat

Large amounts of data can enter the model context.

Programmatic Tool Calling

LLM
 ↓
Generate script
 ↓
Sandbox
 ↓
Get logs
 ↓
Filter + deduplicate
 ↓
Create tickets
 ↓
Print summary
 ↓
LLM

Only the final summary needs to return to the model.

---

13. Combining Progressive Discovery + Programmatic Tool Calling

The two approaches work well together.

Progressive Discovery

The model first:

1. Discovers available tools
2. Identifies the tools it needs
3. Loads their schemas

Programmatic Tool Calling

The model then:

4. Generates one script
5. Calls multiple tools
6. Processes intermediate data in the sandbox
7. Returns only the required final result

Overall:

Tool Discovery
      ↓
Load Required Schemas
      ↓
Generate Program
      ↓
Sandbox Execution
      ↓
Multiple MCP Tool Calls
      ↓
Process Results
      ↓
Final Result
      ↓
LLM

This minimizes both:

• Tool-definition token cost
• Tool-result token cost

and keeps the model’s context focused on reasoning rather than passing intermediate data.

---

14. Key Takeaway

Programmatic Tool Calling is an execution pattern where the LLM writes code to orchestrate MCP tools inside a secure sandbox, while the MCP Host acts as the broker between the sandbox and MCP servers.

The main benefit is that intermediate tool data does not need to pass through the LLM context.

             ┌──────────────┐
             │     LLM      │
             └──────┬───────┘
                    │ Code
                    ▼
             ┌──────────────┐
             │   Sandbox    │
             └──────┬───────┘
                    │
              Function Calls
                    │
                    ▼
             ┌──────────────┐
             │ MCP Host /   │
             │   Broker     │
             └──────┬───────┘
                    │
             ┌──────┴──────┐
             ▼             ▼
        MCP Server A   MCP Server B

Core idea:

Let the model decide what to do, but let the sandbox execute the multi-tool workflow and keep intermediate data out of the model context.

Build servers

Build clients

Build MCP Apps