Smart Crawler
A hosted MCP server that turns e-commerce intelligence into tools your AI can call directly. Point it at a subreddit and get a full growth playbook of top contributors, query competitor catalogs by brand or keyword, pull VOC sentiment from Trustpilot or Google Maps, or run Amazon ASIN reviews through sentiment analysis with listing optimization tips. It also exposes Google Shopping merchant-share rankings and on-demand URL scraping. Reach for this when you're building agents that need to understand product landscapes, mine customer feedback, or automate competitive research without stitching together a dozen scrapers. Connects over streamable-http with Bearer token auth.
- tools
- 7
- auth
- Secret needed
- transport
- HTTP
- last updated
- Jun 8, 2026
Tools
7 toolsPublic tool metadata for what this MCP can expose to an agent.
reddit_subreddit_playbookFull growth playbook for a subreddit's top contributors
Full growth playbook for a subreddit's top contributors
No parameter schema in public metadata yet.
search_competitor_productsSearch competitor catalog by brand / country / keyword / price
Search competitor catalog by brand / country / keyword / price
No parameter schema in public metadata yet.
get_voc_reviewsConsumer VOC reviews from Trustpilot / Reviews.io / Google Maps
Consumer VOC reviews from Trustpilot / Reviews.io / Google Maps
No parameter schema in public metadata yet.
amazon_voc_reportAmazon ASIN reviews + AI VOC analysis (pain points, listing tips)
Amazon ASIN reviews + AI VOC analysis (pain points, listing tips)
No parameter schema in public metadata yet.
competitor_landscapeGoogle Shopping merchant-share ranking for a keyword
Google Shopping merchant-share ranking for a keyword
No parameter schema in public metadata yet.
scrape_urlOne-URL scrape with short-lived agent memory
One-URL scrape with short-lived agent memory
No parameter schema in public metadata yet.
query_warehouseWarehouse-first natural-language query (0 credits)
Warehouse-first natural-language query (0 credits)
No parameter schema in public metadata yet.
Sponsors
Agent CaffeineSponsoredClose your lid, come back to finished work
SnugSponsoredThe clipboard manager for the AI eraPromote your product hereReach 125k+ visitors monthly
WaxSponsoredBlock anything with supervision, without erasing your iPhonePromote your product hereReach 125k+ visitors monthly
knobsSponsoredSee your site on any phone, any day, in any languageReadme
smart-crawler MCP
E-commerce competitor intelligence, VOC review sentiment, Amazon ASIN analysis, Google Shopping landscape, on-demand URL scraping, and Reddit subreddit growth playbooks — exposed to AI agents as a hosted Model Context Protocol server.
- Endpoint:
https://smartcrawler.io/mcp/(streamable-http) - Auth:
Authorization: Bearer sck_...— generate a key in the smart-crawler console - Registry:
io.github.shulex-voc/smart-crawler
This repository hosts the public MCP manifest for the smart-crawler service. It is the discovery/connection entry point — the service itself is hosted.
Connect
Claude Code
export SMARTCRAWLER_API_KEY=sck_...
claude mcp add --scope user --transport http smart-crawler \
https://smartcrawler.io/mcp/ \
--header "Authorization: Bearer $SMARTCRAWLER_API_KEY"
--scope user makes the server visible in every project. Without it,
claude mcp add defaults to local scope (only the current directory), a common
"I added it but /mcp is empty" pitfall. Verify with claude mcp list or /mcp.
Details
- Repository
- github.com/shulex-voc/smart-crawler
- Categories
- Search & Web Crawling, Finance & Commerce
- Transport
- HTTP
- Auth
- Required
- Registry
- MCP registry, active
Related Search & Web Crawling MCP servers
View all
Airweave Searchairweave-ai/airweaveMCP server for searching Airweave collections with natural language queries.6.4Kstars
Paper Searchopenags/paper-search-mcpA MCP for searching and downloading academic papers from multiple sources like arXiv, PubMed, bioRxiv, etc.1.5Kstars
Everything Searchmamertofabian/mcp-everything-searchFast file search: Everything SDK on Windows, Spotlight (mdfind) on macOS, locate/plocate on Linux.367stars
G-Searchjae-jae/g-search-mcpA powerful MCP server for Google search that enables parallel searching with multiple keywords simultaneously.263stars