MCP 服务器测试器是一款用于自动测试 MCP 服务器的工具。

为什么是这个工具？

构建可靠且稳定且符合规范的MCP服务器比预期更难：

• Inspector 工具帮助构建和测试初始服务器，但无法防止回归
• 工具在直接测试中运行良好，但大型语言模型在现实中使用它们时不正确且不一致
• 大多数MCP SDK并不完全符合规范，很难判断缺失了什么
• 规范快速发展，难以跟上最佳实践

mcp-server-tester 是一款与实现无关的 CLI 测试工具，提供三个聚焦命令，帮助解决这些核心问题：

• tools - 运行测试，验证工具在直接MCP调用下是否正确工作
• evals - 运行测试，验证大型语言模型是否能持续正确理解并正确使用你的工具
• compliance - 随着协议演进自动验证服务器规范 （进行中）

---

命令

tools - 直接MCP工具测试

测试工具是否能正常运行，直接调用API：

样本输出：

evals - 大型语言模型集成测试

MCP Server Tester

The MCP Server Tester is a tool for automated testing of MCP servers.

Why this tool?

Building reliable MCP servers that work consistently and are spec-compliant is harder than it should be:

• the Inspector tool helps to build and test the intial server, but won’t protect you from regressions
• tools works perfectly in direct testing, but LLMs use them incorrectly and inconsistently in the real world
• most MCP SDKs aren’t fully spec-compliant, and it’s hard to know what’s missing
• the spec evolves rapidly, making it hard to stay current with best practices

The mcp-server-tester is a implementation-agnostic CLI testing tool that provides three focused commands that help solve these core issues:

• tools - Run tests that verify your tools work correctly with direct MCP calls
• evals - Run tests that verify LLMs can consistently understand and use your tools correctly
• compliance - Automated server spec validation as the protocol evolves (Work in Progress)

---

Commands

tools - Direct MCP Tool Testing

Test your tools work correctly with direct API calls:

npx mcp-server-tester tools tool-tests.yaml --server-config filesystem-server-config.json

Sample output:

✅ Tool Discovery: Found 3 tools (read_file, write_file, delete_file)
✅ Write file successfully
✅ Read file returns content
✅ Write and read workflow
❌ Handle invalid path - Expected error containing 'permission' but got 'access denied'
✅ Delete file cleanup

Results: 5/6 tests passed

evals - LLM Integration Testing

Test that LLMs can discover and use your tools effectively:

npx mcp-server-tester evals eval-tests.yaml --server-config filesystem-server-config.json

Sample output:

🤖 Testing with claude-3-5-haiku-latest
✅ LLM can create and read files
✅ File manipulation workflow
✅ Error handling test
❌ Documentation query task: FAILED
    Prompt: "How do I search for configuration files in the project?"
    • Required tool 'search_docs' was not called (actual calls: list_files)
❌ Lists all available tools: FAILED
    Prompt: "Please list all available file management tools you have access to."
    • LLM judge failed: Score: 0.3/1.0 (threshold: 0.8). Assistant did not provide complete tool list.

Results: 3/5 tests passed

Note: Requires ANTHROPIC_API_KEY environment variable.

compliance - MCP Specification Compliance Testing (WIP)

Test that your server is spec compliant:

npx mcp-server-tester compliance --server-config filesystem-server-config.json

See the Compliance Command (WIP) section below for details and example output.

Quick Start

1. Create server config (filesystem-server-config.json):

   {
     "mcpServers": {
       "filesystem": {
         "command": "node",
         "args": ["./filesystem-server.js"]
       }
     }
   }
   

2. Create tool and eval test files manually: