Threads Scraper MCP Server

Give AI assistants structured, read-only access to public Threads.net data (profiles + posts).

Uses Playwright headless Chrome to bypass Threads’ login wall - no authentication needed.

Example Use Cases

Analyze posting strategies and engagement metrics for any public Threads profile:

---

Claude Desktop Setup

1. Install Dependencies

# Clone the repository
git clone https://github.com/dkyazzentwatwa/threads-mcp-crawler.git
cd threads-mcp-crawler

# Create virtual environment
python3 -m venv .venv
source .venv/bin/activate

# Install package
pip install -e .

# Install Playwright browser (required)
playwright install chromium

2. Add to Claude Desktop Config

Edit ~/Library/Application Support/Claude/claude_desktop_config.json:

{
  "mcpServers": {
    "threads-scraper": {
      "command": "/path/to/threads-mcp-crawler/.venv/bin/python3",
      "args": [
        "/path/to/threads-mcp-crawler/run_mcp_server.py"
      ]
    }
  }
}

Note: Replace /path/to/threads-mcp-crawler with your actual installation path.

3. Restart Claude Desktop

The tools will be available immediately. Test with:

"Get profile info for @zuck"

---

Available Tools

get_profile

Get profile information including bio, follower counts, verification status.

"Get profile info for @meta"

Returns: handle, display_name, bio, follower_count, following_count, thread_count, is_verified, is_private, url

get_profile_threads

Fetch recent threads from a profile.

"Show me the last 20 posts from @zuck"

Parameters:

• handle: Username
• limit: 0 for all (default 10, capped by THREADS_MAX_PAGINATION_REQUESTS)

Returns: List of threads with text, engagement (likes/replies), timestamps, media URLs

get_thread

Get single thread by URL or ID.

"Get details for https://threads.net/@zuck/post/ABC123"

get_thread_replies

Get direct replies to a thread.

"Show me replies to this thread"

search_threads

Search for threads by query or hashtag.

"Search for threads about AI"
"Find posts with #technology"

Parameters:

• query: Search text or hashtag
• limit: Max results (default 10)

analyze_profile_threads

Comprehensive analytics on recent threads.

"Analyze @zuck's recent posts - what topics and sentiment?"

Parameters:

• handle: Username
• post_limit: 5-50 (default 20)

Returns:

• thread_count: Number of threads analyzed
• posting_cadence_days: Average days between posts
• avg_engagement: Average likes, replies, reposts
• sentiment: VADER scores (positive, negative, neutral, compound)
• keywords: Top 15 keywords and hashtags

---

Example Prompts

Profile Research

"Show me @zuck's profile and recent threads"
"What topics is @meta posting about?"
"How many followers does @instagram have?"

Content Analysis

"Analyze sentiment in @brand's recent posts"
"What hashtags does @creator use most?"
"What's the average engagement for @influencer?"

Search & Discovery

"Search for threads about artificial intelligence"
"Find posts mentioning 'startup'"
"Show me #tech threads"

---

Configuration

Environment Variables

# Throttle delay between requests (default: 2 seconds)
export THREADS_THROTTLE=2.0

# Cache TTL (default: 600 seconds / 10 minutes)
export THREADS_CACHE_TTL=600

---

How It Works

Browser-Based Scraping

1. Playwright launches headless Chrome
2. Navigates to Threads.net pages
3. Intercepts GraphQL responses for structured data
4. Falls back to HTML parsing when needed

Why Playwright?

Threads.net requires login for all content. Playwright with headless Chrome:

• Executes JavaScript (renders React app)
• Bypasses bot detection
• Captures GraphQL API responses
• No cookies/authentication needed

Data Pipeline

Request → Playwright → GraphQL Interception → Pydantic Models → JSON Response

Caching & Throttling

• 2-second delay between requests (prevents rate limiting)
• 10-minute cache (reduces repeat requests)
• 256 item LRU cache (memory efficient)

---

Limitations

1. Public content only - No private profiles
2. Pagination limits - Up to ~1000 threads per profile (controlled by THREADS_MAX_PAGINATION_REQUESTS)
3. Search results vary - May require authentication for full results
4. Rate limits - Aggressive use may trigger Threads rate limiting
5. Structure changes - Meta may update HTML/GraphQL schemas without notice

---

Troubleshooting

“Profile not found”

• Check handle spelling
• Profile may be private or suspended

Empty search results

• Try different keywords
• Search scrapes HTML (less reliable than GraphQL)

Slow responses

• First request launches browser (~3-5 seconds)
• Subsequent requests use existing browser (~2-3 seconds)

Rate limited

• Increase THREADS_THROTTLE to 5+ seconds
• Wait a few minutes before retrying

---

Project Structure

src/threads_scraper_mcp/
├── client.py         # Playwright client + GraphQL interception
├── models.py         # Pydantic data models
├── analysis.py       # Sentiment + keyword analytics
├── mcp_server.py     # MCP tool handlers
├── cache.py          # TTL caching
└── settings.py       # Configuration

---

Development

Manual Testing

source .venv/bin/activate
python3 -c "
from threads_scraper_mcp.client import ThreadsPublicClient
client = ThreadsPublicClient()
profile = client.get_profile('zuck')
print(profile.model_dump_json(indent=2))
client.close()
"

Run MCP Server Directly

python run_mcp_server.py

---

License

MIT License - Use at your own risk

Disclaimer

This tool is not affiliated with Meta or Threads. Scraping may violate Threads Terms of Service. Use responsibly for personal research only.