mcp-load-lab

A load and latency harness for Model Context Protocol (MCP) servers.

Most MCP tooling checks whether a server is correct: does it follow the spec, is its tool surface stable, is it safe to deploy. mcp-load-lab answers a different question: how does the server behave under concurrent load? It drives real tools/call traffic at a chosen concurrency, measures latency percentiles and throughput, and fails your CI build when a server misses its performance budget.

It has no required dependencies for the stdio transport (Python standard library only). HTTP transport adds a single optional dependency.

Install

pip install mcp-load-lab            # stdio transport, zero deps
pip install "mcp-load-lab[http]"    # adds streamable-HTTP support (httpx)

Or from a clone:

pip install -e .

Quick start

Every example below works against the bundled demo server in examples/, so you can try the tool before pointing it at your own.

List a server’s tools:

mcp-load-lab tools --stdio "python examples/echo_server.py"

Ad-hoc load run, no config file needed:

mcp-load-lab quick \
  --stdio "python examples/echo_server.py" \
  --tool sleep --arg ms=20 \
  --concurrency 32 --duration 5 --warmup 1

phase    conc  reqs  rps     ok%    mean  p50   p90   p95   p99   max
-------  ----  ----  ------  -----  ----  ----  ----  ----  ----  ----
warmup*  32    1485  1454.5  100.0  21.7  21.6  22.4  22.5  23.5  23.6
load     32    4512  1498.3  100.0  21.3  21.3  21.9  22.1  22.4  22.9

(latencies in ms; * = warmup, excluded from gates)

Scenario files

For repeatable runs and CI, describe the whole test in a JSON or TOML file:

{
  "name": "echo-sleep-ramp",
  "transport": {
    "type": "stdio",
    "command": "python",
    "args": ["examples/echo_server.py"]
  },
  "calls": [
    { "tool": "sleep", "arguments": { "ms": 20 }, "weight": 3 },
    { "tool": "echo",  "arguments": { "text": "ping" }, "weight": 1 }
  ],
  "phases": [
    { "name": "warmup", "concurrency": 4,  "duration_s": 1, "warmup": true },
    { "name": "c8",     "concurrency": 8,  "duration_s": 3 },
    { "name": "c32",    "concurrency": 32, "duration_s": 3 },
    { "name": "c64",    "concurrency": 64, "duration_s": 3 }
  ],
  "timeout_s": 10,
  "gates": {
    "max_p95_ms": 120,
    "max_error_rate": 0.01,
    "min_throughput_rps": 200
  }
}

Run it:

mcp-load-lab run examples/scenario.json --json report.json

• calls are chosen per request by weight (weighted random). Use this to model a realistic mix of tools.
• phases run in order. A warmup phase is measured but excluded from the gates. Describe a concurrency ramp by listing several phases, each with a higher concurrency.
• gates are optional thresholds. If any gate fails, the aggregate of all non-warmup phases is reported as FAILED and the process exits non-zero.

Transports

• stdio launches your server as a subprocess and speaks newline-delimited JSON-RPC over its pipes. Concurrent requests are multiplexed over the single connection with distinct ids, so a server that handles requests concurrently shows real concurrency.

  { "type": "stdio", "command": "python", "args": ["server.py"], "env": {}, "cwd": "." }
  

• http POSTs JSON-RPC to a streamable-HTTP endpoint using a pooled client (needs the http extra). Session ids returned on initialize are reused.

  { "type": "http", "url": "http://localhost:8000/mcp", "headers": {} }
  

CI gating

mcp-load-lab exits 0 when all gates pass and 1 when any gate fails, so it drops straight into a pipeline as a performance regression check:

- name: MCP performance budget
  run: mcp-load-lab run perf/scenario.json --json perf/report.json

Gates can also be set or overridden on the command line:

mcp-load-lab run perf/scenario.json \
  --max-p95 150 --max-error-rate 0.01 --min-rps 300

Metrics

Per phase and for the non-warmup aggregate the harness reports request count, throughput (requests/second over wall-clock time), success rate, and latency min / mean / p50 / p90 / p95 / p99 / max in milliseconds. Percentiles use linear interpolation over the sorted per-request latencies. Failures are broken out by kind: tool_error (server returned isError), rpc_error (JSON-RPC error), timeout, and exception.

Demo server tools

examples/echo_server.py is a dependency-free async MCP server exposing four tools chosen to exercise different load shapes: echo (trivial), sleep(ms) (I/O-bound), fib(n) (CPU-bound), and flaky(rate) (fails with a given probability, for testing the error path and error-rate gates).

Load model

The engine is closed-loop: each phase runs a fixed number of concurrent workers, and each worker issues one call, waits for the response, and immediately issues the next until the phase’s duration_s or requests budget is spent. This measures how a server responds at a specific concurrency level. Open-loop (fixed target RPS) pacing is on the roadmap.

License

Apache-2.0. See LICENSE.