Xcrawl Crawl

This is your interface to XCrawl's full-site crawling API. You get async crawl orchestration with depth limits, regex path filtering, and multi-format output (markdown, links, JSON extraction via prompt or schema). It reads your API key from a local config file at ~/.xcrawl/config.json and uses only curl and node for execution, no Python dependencies. The skill returns raw API responses by default, so you handle the data structure yourself. Useful when you need to map an entire documentation site or product catalog before scraping specific pages. XCrawl runs on a credit system with 1000 free credits to start. The workflow is standard: POST to create a crawl job, poll GET until status hits completed, then parse the data array.

skills.sh installs: 215
GitHub stars: 449
last updated: Aug 21, 2026

View on GitHub

Install

$ npx -y skills add xcrawl-api/xcrawl-skills --skill xcrawl-crawl --agent claude-code

Installs into .claude/skills of the current project.

Files

XCrawl Crawl

Overview

This skill orchestrates full-site or scoped crawling with XCrawl Crawl APIs. Default behavior is raw passthrough: return upstream API response bodies as-is.

Required Local Config

Before using this skill, the user must create a local config file and write XCRAWL_API_KEY into it.

Path: ~/.xcrawl/config.json

{
  "XCRAWL_API_KEY": "<your_api_key>"
}

Read API key from local config file only. Do not require global environment variables.

Credits and Account Setup

Using XCrawl APIs consumes credits. If the user does not have an account or available credits, guide them to register at https://dash.xcrawl.com/. After registration, they can activate the free 1000 credits plan before running requests.