🕷️ Crawl4AI MCP Server

MCP (Model Context Protocol) server for Crawl4AI - Universal web crawling and data extraction for AI agents.

Integrate powerful web scraping capabilities into Claude, ChatGPT, and any MCP-compatible AI assistant.

📑 Table of Contents

• 🚀 Quick Start
• 🐳 Docker Usage
• 🛠️ Available Tools
• ⚙️ Configuration
• 🤝 Contributing
• 📄 License

🚀 Quick Start

NPM Installation (Recommended)

# Install globally
npm install -g crawl4ai-mcp-sse-stdio

# Run in different modes
npx crawl4ai-mcp --stdio --endpoint https://your-crawl4ai-server.com
npx crawl4ai-mcp --sse --port 3001 --endpoint https://your-crawl4ai-server.com
npx crawl4ai-mcp --http --port 3000 --endpoint https://your-crawl4ai-server.com

# With optional bearer token
npx crawl4ai-mcp --stdio --endpoint https://your-crawl4ai-server.com --bearer-token your-token

With Claude Desktop

Add to your claude_desktop_config.json:

{
  "mcpServers": {
    "crawl4ai": {
      "command": "npx",
      "args": [
        "crawl4ai-mcp-sse-stdio",
        "--stdio",
        "--endpoint", "https://your-crawl4ai-server.com",
        "--bearer-token", "your-optional-token"
      ]
    }
  }
}

🐳 Docker Usage

Docker Hub

Official Docker image available on Docker Hub:

# Pull and run the official Docker image
docker pull stgmt/crawl4ai-mcp:latest
docker run -p 3000:3000 stgmt/crawl4ai-mcp:latest

Docker Compose

Create a docker-compose.yml:

version: '3.8'
services:
  crawl4ai-mcp:
    image: stgmt/crawl4ai-mcp:latest
    ports:
      - "3000:3000"
    environment:
      - CRAWL4AI_ENDPOINT=https://your-crawl4ai-server.com
      - CRAWL4AI_BEARER_TOKEN=your-optional-token
    restart: unless-stopped

Run with:

docker-compose up -d

🛠️ Available Tools

1. crawl - Full Web Crawling

Extract complete content from any webpage.

{
  "name": "crawl",
  "arguments": {
    "url": "https://example.com",
    "wait_for": "css:.content",
    "timeout": 30000
  }
}

2. md - Markdown Extraction

Get clean markdown content from webpages.