MMCPMarkets
DM

dennisgl/mcp-playwright-scraper

开发工具

本页由 AI 翻译，技术细节请以原文为准。 查看原文

11 stars 8 forks 质量 67 趋势 18Apache-2.0
Repository Install

A Model Context Protocol (MCP) server that scrapes web content and converts it to Markdown.

概览

A Model Context Protocol (MCP) server that scrapes web content and converts it to Markdown. This MCP server provides a simple tool for scraping web content and converting it to Markdown format. It uses: - : For headless browser automation to handle modern web pages including JavaScript-heavy sites - : For HTML parsing and cleanup - : For high-quality HTML to Markdown conversion - scrape_to_markdown: Scrapes content from a URL and converts it to Markdown - Required parameter: url (string) - The URL to scrape - Optional parameter: verify_ssl (boolean) - Whether to verify SSL certificates (default: true) When using uv no specific installation is needed. We will use uvx to directly run mcp-playwright-scraper. Alternatively you can install mcp-playwright-scraper via pip: After installation, you can run it as a script using: - Python 3.11 or higher - Playwright browser dependencies - Pandoc (optional, will be automatically installed by pypandoc if possible)

README

以下为原文 README 节选，完整内容请查看原文。

mcp-playwright-scraper

A Model Context Protocol (MCP) server that scrapes web content and converts it to Markdown.

Overview

This MCP server provides a simple tool for scraping web content and converting it to Markdown format. It uses:

  • Playwright: For headless browser automation to handle modern web pages including JavaScript-heavy sites
  • BeautifulSoup: For HTML parsing and cleanup
  • Pypandoc: For high-quality HTML to Markdown conversion

Tools

The server implements a single tool:

  • scrape_to_markdown: Scrapes content from a URL and converts it to Markdown
    • Required parameter: url (string) - The URL to scrape
    • Optional parameter: verify_ssl (boolean) - Whether to verify SSL certificates (default: true)

Installation

When using uv no specific installation is needed. We will use uvx to directly run mcp-playwright-scraper.

Using PIP

Alternatively you can install mcp-playwright-scraper via pip:

pip install mcp-playwright-scraper

After installation, you can run it as a script using:

python -m mcp_playwright_scraper

Prerequisites

  • Python 3.11 or higher
  • Playwright browser dependencies
  • Pandoc (optional, will be automatically installed by pypandoc if possible)

After installation, you need to install Playwright browser dependencies:

playwright install --with-deps chromium

Configuration

Usage with Claude Desktop

Add this to your claude_desktop_config.json:

Usage with Claude Code

# Basic syntax
$ claude mcp add mcp-playwright-scraper -- uvx mcp-playwright-scraper

# Alternatively, with pip installation
$ claude mcp add mcp-playwright-scraper -- python -m mcp_playwright_scraper

Usage with Zed

Add to your Zed settings.json:

"context_servers": {
  "mcp-playwright-scraper": {
    "command": "python",
    "args": ["-m", "mcp_playwright_scraper"]
  }
},
在 GitHub 查看完整 README

安装

uvx mcp-playwright-scraper
新手？先看 mcpskill.org 教程 →

配置

"mcpServers": { "mcp-playwright-scraper": { "command": "uvx", "args": ["mcp-playwright-scraper"] } }