MMCPMarkets
RM

rayss868/mcp-web-curl

Developer tools

Эта страница переведена автоматически. Технические детали смотрите в оригинале. Открыть оригинал

11 stars 3 forks Качество 81 Тренд 22MIT
Repository Install

Lightweight MCP server and CLI that lets AI fetch and process web content or APIs with Puppeteer, smart commands, search tools, and reliable downloads.

Обзор

🚀 🛠️ Built with Node.js & TypeScript (Node.js v18+ required) Click here to watch the demo video directly in your browser. If your platform supports it, you can also download and play demo/demo.mp4 directly. Your browser does not support the video tag. See CHANGELOG.md for a complete history of updates and new features. is a powerful tool for fetching and extracting text content from web pages and APIs. Use it as a standalone CLI or as an MCP (Model Context Protocol) server. Web-curl leverages Puppeteer for robust web scraping and supports advanced features such as resource blocking, custom headers, authentication, and External Search API integration. - : Full control over Chromium via Puppeteer (click, type, scroll, hover, key presses). - : Browser profiles are now always persistent. Login sessions, cookies, and cache are automatically saved in a local user_data/ directory.

README

Показан фрагмент оригинального README. Открыть оригинал.

Web-curl

Developed by Rayss

🚀 Open Source Project
🛠️ Built with Node.js & TypeScript (Node.js v18+ required)

🎬 Demo Video

Click here to watch the demo video directly in your browser.

If your platform supports it, you can also download and play demo/demo.mp4 directly.

📚 Table of Contents

📝 Changelog / Update History

See CHANGELOG.md for a complete history of updates and new features.

📝 Overview

Web-curl is a powerful tool for fetching and extracting text content from web pages and APIs. Use it as a standalone CLI or as an MCP (Model Context Protocol) server. Web-curl leverages Puppeteer for robust web scraping and supports advanced features such as resource blocking, custom headers, authentication, and External Search API integration.

✨ Features

🚀 Deep Research & Automation (v1.4.2)

  • Advanced Browser Automation: Full control over Chromium via Puppeteer (click, type, scroll, hover, key presses).
  • Always-On Session Persistence: Browser profiles are now always persistent. Login sessions, cookies, and cache are automatically saved in a local user_data/ directory.
  • Token-Efficient Snapshots (available via the hidden browser_snapshot handler):
    • Accessibility Tree: Clean, structured snapshots instead of messy HTML.
    • HTML Slice Mode: Raw HTML with startIndex/endIndex for safe chunking when needed.
Открыть полный README на GitHub

Установка

This server does not publish a one-line install command.

Open the repository installation guide Новичок в MCP? Начните с руководства на mcpskill.org →

Конфигурация

{ "mcpServers": { "web-curl": { "command": "node", "args": [ "build/index.js" ], "disabled": false, "alwaysAllow": [ "extract", "crawl", "agent", "research", "browser_configure", "browser_close", "multi_search", "fetch_api", "download_file", "parse_document" ], "env": { "SEARCH_BASE_URL": "https://example.com/v1/search", "SEARCH_MODEL": "search-combo", "SEARCH_API_KEY": "YOUR_EXTERNAL_SEARCH_API_KEY", "APIKEY_GOOGLE_SEARCH": "YOUR_GOOGLE_API_KEY", "CX_GOOGLE_SEARCH": "YOUR_CX_ID" } } } }