A Playwright-based Python tool that bypasses Google's anti-bot mechanisms to execute searches, serving as both a command-line tool and MCP server to provide real-time search capabilities for AI...
概览
A Playwright-based Python tool that bypasses search engine anti-scraping mechanisms to execute Google searches and extract results. It can be used directly as a command-line tool or as a Model Context Protocol (MCP) server to provide real-time search capabilities to AI assistants like Claude. - : No need to rely on paid search engine results API services, all searches are executed locally - : - Intelligent browser fingerprint management that simulates real user behavior - Automatic saving and restoration of browser state to reduce verification frequency - Smart headless/headed mode switching, automatically switching to headed mode when verification is needed - Randomization of device and locale settings to reduce detection risk - : Ability to fetch the raw HTML of search result pages (with CSS and JavaScript removed) for analysis and debugging when Google's page structure changes - : Automatically captures and saves a full-page screenshot when saving HTML content - : Provides...
README
以下为原文 README 节选，完整内容请查看原文。
Google Search Tool
A Playwright-based Python tool that bypasses search engine anti-scraping mechanisms to execute Google searches and extract results. It can be used directly as a command-line tool or as a Model Context Protocol (MCP) server to provide real-time search capabilities to AI assistants like Claude.
Key Features
- Local SERP API Alternative: No need to rely on paid search engine results API services, all searches are executed locally
- Advanced Anti-Bot Detection Bypass Techniques:
- Intelligent browser fingerprint management that simulates real user behavior
- Automatic saving and restoration of browser state to reduce verification frequency
- Smart headless/headed mode switching, automatically switching to headed mode when verification is needed
- Randomization of device and locale settings to reduce detection risk
- Raw HTML Retrieval: Ability to fetch the raw HTML of search result pages (with CSS and JavaScript removed) for analysis and debugging when Google’s page structure changes
- Page Screenshot: Automatically captures and saves a full-page screenshot when saving HTML content
- MCP Server Integration: Provides real-time search capabilities to AI assistants like Claude without requiring additional API keys
- Completely Open Source and Free: All code is open source with no usage restrictions, freely customizable and extensible
- Python Native: Built with Python for better performance and easier deployment
Technical Features
- Developed with Python 3.8+, providing excellent performance and wide compatibility
- Browser automation based on Playwright, supporting multiple browser engines
- Command-line parameter support for search keywords
- MCP server support for AI assistant integration
- Returns search results with title, link, and snippet
- Option to retrieve raw HTML of search result pages for analysis
- JSON format output
- Support for both headless and headed modes (for debugging)
- Detailed logging output
- Robust error handling
- Browser state saving and restoration to effectively avoid anti-bot detection
- Anti-bot protection mechanisms at multiple levels
Installation
# Install from source
git clone https://github.com/iwanghc/mcp_web_search.git
cd mcp_web_search
# Install Python dependencies
pip install -r requirements.txt
# Install Playwright browsers
playwright install chromium
Usage
Command Line Tool
# Direct command line usage
python cli.py "search keywords"
# Using command line options
python cli.py --limit 5 --timeout 30000 "search keywords"
# Get raw HTML of search result page
python cli.py --get-html "search keywords"
# Get HTML and save to file
python cli.py --get-html --save-html "search keywords"
MCP Server
# Configure model API KEY and other information in dotenv.env
# USE MCP server
python -m mcp_integration.client
Command Line Options
安装
This server does not publish a one-line install command.
Open the repository installation guide 新手？先看 mcpskill.org 教程 →配置
{
"mcpServers": {
"google-search": {
"command": "python",
"args": ["mcp_server_simple.py"]
}
}
}