BS

bamchi/scrapi-mcp-server

开发工具

本页由 AI 翻译,技术细节请以原文为准。 查看原文

5 stars 4 forks 质量 82 趋势 17MIT

🔗 MCP server that converts URLs to clean Markdown/Text for LLM agents

概览

MCP server that converts URLs to clean Markdown/Text for LLM agents — Built on 8+ years of web scraping expertise, 1,900+ production crawlers, and battle-tested anti-bot handling. An MCP (Model Context Protocol) server that lets AI agents fetch and read web pages. Simply give it a URL, and it returns clean, LLM-ready content — fast. AI can't read web pages directly "Summarize this article" just works ✨ - 🌐 : Preserves headings, lists, links - 📄 : Plain text extraction - 🏷️ : Title, author, date, images - 🧹 : No ads, no navigation, no scripts - ⚡ : Works with SPAs - 💳 : Credit tracking, subscription management, usage analytics (MCP keys) - 🔄 : 429 rate limit responses automatically retried with Retry-After - 🌍 : Stdio (npx) + Streamable HTTP for flexible deployment Scrapi MCP Server supports two transport modes: - Scrapi MCP account (separate from the main Scrapi account) - Claude Desktop, Cline, or Cursor installed - Node.js 20+ No installation needed.

README

以下为原文 README 节选,完整内容请查看原文。

MCP server that converts URLs to clean Markdown/Text for LLM agents

한국어 · Website · Dashboard

⚡ Fast & Reliable — Built on 8+ years of web scraping expertise, 1,900+ production crawlers, and battle-tested anti-bot handling.

What is this?

An MCP (Model Context Protocol) server that lets AI agents fetch and read web pages. Simply give it a URL, and it returns clean, LLM-ready content — fast.

Before: AI can’t read web pages directly
After: “Summarize this article” just works ✨


Features

  • 🌐 URL → Markdown: Preserves headings, lists, links
  • 📄 URL → Text: Plain text extraction
  • 🏷️ Metadata: Title, author, date, images
  • 🧹 Clean Output: No ads, no navigation, no scripts
  • ⚡ JavaScript Rendering: Works with SPAs
  • 💳 Built-in Billing: Credit tracking, subscription management, usage analytics (MCP keys)
  • 🔄 Auto-Retry: 429 rate limit responses automatically retried with Retry-After
  • 🌍 Dual Transport: Stdio (npx) + Streamable HTTP for flexible deployment

Transport Modes

Scrapi MCP Server supports two transport modes:

Mode Best For Node.js Required
Stdio Claude Desktop, Cursor, Cline, Claude Code Yes (auto via npx)
Streamable HTTP All clients, Node.js-free environments No

Prerequisites

  • Scrapi MCP account (separate from the main Scrapi account)
  • Claude Desktop, Cline, or Cursor installed
  • Node.js 20+

Installation

No installation needed. Just configure your MCP client to use npx.

{
  "mcpServers": {
    "scrapi": {
      "command": "npx",
      "args": ["-y", "@scrapi.ai/mcp-server"],
      "env": {
        "SCRAPI_API_KEY": "your-api-key"
      }
    }
  }
}

Tip: You can also pass the API key via CLI argument instead of env var:

"args": ["-y", "@scrapi.ai/mcp-server", "--api-key", "your-api-key"]

See Step 2 for where to put this configuration.

Option B: Install from Source

# Clone the repository
在 GitHub 查看完整 README

安装

npx -y @scrapi.ai/mcp-server
新手?先看 mcpskill.org 教程 →

配置

{ "mcpServers": { "scrapi": { "command": "npx", "args": ["-y", "@scrapi.ai/mcp-server"], "env": { "SCRAPI_API_KEY": "your-api-key" } } } }