为MCP（模型上下文协议）服务器提供超快速语义工具过滤，使用嵌入相似度。将工具上下文从1000+工具缩减到最相关的10-20个工具，时间不到10毫秒。

特色

• ⚡ 闪电般快：

用MCP服务器初始化过滤器。这会预先计算并缓存所有工具嵌入。

注意：启动时调用一次。这是一个异步操作，可能需要几秒钟，具体取决于工具数量。

filter(input: FilterInput, options?: FilterOptions): Promise

根据输入上下文使用过滤工具。

输入类型：

选项（全部可选，覆盖默认）：

退货：

getStats()

获取滤波器状态的统计数据。

clearCache()

清除上下文嵌入缓存。

性能优化

内置优化

该库开箱即用包含多项性能优化：

1. ** 🚀 环展开点积** - 通过CPU流水线优化，向量相似度计算速度提升6-8倍
2. ** 📊 Smart Top-K Selection** - 混合算法为典型工作负载使用快速内置排序，500+ 工具则切换为基于堆的选择

@portkey-ai/mcp-tool-filter

Ultra-fast semantic tool filtering for MCP (Model Context Protocol) servers using embedding similarity. Reduce your tool context from 1000+ tools down to the most relevant 10-20 tools in under 10ms.

Features

• ⚡ Lightning Fast: <10ms filtering latency for 1000+ tools with built-in optimizations
• 🚀 Performance Optimized: 6-8x faster dot product, smart top-K selection, true LRU cache
• 🎯 Semantic Understanding: Uses embeddings for intelligent tool matching
• 📦 Zero Dependencies on Runtime: Only requires an embedding provider API
• 🔄 Flexible Input: Accepts chat completion messages or raw strings
• 💾 Smart Caching: Caches embeddings and context for optimal performance
• 🎛️ Configurable: Tune scoring thresholds, top-k, and always-include tools
• 📊 Performance Metrics: Built-in timing for optimization

Installation

npm install @portkey-ai/mcp-tool-filter

Quick Start

import { MCPToolFilter } from '@portkey-ai/mcp-tool-filter';

// 1. Initialize the filter (choose embedding provider)

// Option A: Local Embeddings (RECOMMENDED for low latency < 5ms)
const filter = new MCPToolFilter({
  embedding: {
    provider: 'local',
  }
});

// Option B: API Embeddings (for highest accuracy)
const filter = new MCPToolFilter({
  embedding: {
    provider: 'openai',
    apiKey: process.env.OPENAI_API_KEY,
  }
});

// 2. Load your MCP servers (one-time setup)
await filter.initialize(mcpServers);

// 3. Filter tools based on context
const result = await filter.filter(
  "Search my emails for the Q4 budget discussion"
);

// 4. Use the filtered tools in your LLM request
console.log(result.tools); // Top 20 most relevant tools
console.log(result.metrics.totalTime); // e.g., "2ms" for local, "500ms" for API

Embedding Provider Options

Local Embeddings (Recommended)

Pros:

• ⚡ Ultra-fast: 1-5ms latency
• 🔒 Private: No data sent to external APIs
• 💰 Free: No API costs
• 🌐 Offline: Works without internet

Cons:

• Slightly lower accuracy than API models
• First initialization downloads model (~25MB)

const filter = new MCPToolFilter({
  embedding: {
    provider: 'local',
    model: 'Xenova/all-MiniLM-L6-v2', // Optional: default model
    quantized: true, // Optional: use quantized model for speed (default: true)
  }
});

Available Models:

• Xenova/all-MiniLM-L6-v2 (default) - 384 dimensions, very fast
• Xenova/all-MiniLM-L12-v2 - 384 dimensions, more accurate
• Xenova/bge-small-en-v1.5 - 384 dimensions, good balance
• Xenova/bge-base-en-v1.5 - 768 dimensions, higher quality

Performance:

• Initialization: 100ms-4s (one-time, downloads model)
• Filter request: 1-5ms
• Cached request: <1ms

API Embeddings

For highest accuracy, use OpenAI or other API providers:

const filter = new MCPToolFilter({
  embedding: {
    provider: 'openai',
    apiKey: process.env.OPENAI_API_KEY,