Web Fetch

This is a three-tier fetch system that tries to get you clean markdown from any URL without spinning up a browser. It checks if the endpoint serves native markdown first, falls back to selector-based HTML parsing with html2markdown for known sites like MDN or Anthropic's docs, and keeps a Bun script as the nuclear option when neither works. The selector table is short but covers the essentials, and the troubleshooting section actually tells you what to do when you hit client-rendered content (spoiler: you can't curl it). Honest take: it won't save you from JavaScript-heavy SPAs, but for documentation and articles it beats copy-pasting from your browser.

skills.sh installs: 929
GitHub stars: 51
last updated: Jul 14, 2026

View on GitHub

Install

$ npx -y skills add 0xbigboss/claude-code --skill web-fetch --agent claude-code

Installs into .claude/skills of the current project.

Files

Web Content Fetching

Fetch web content in this order:

Prefer markdown-native endpoints (content-type: text/markdown)
Use selector-based HTML extraction for known sites
Use the bundled Bun fallback script when selectors fail

Prerequisites

Verify required tools before extracting:

command -v curl >/dev/null || echo "curl is required"
command -v html2markdown >/dev/null || echo "html2markdown is required for HTML extraction"
command -v bun >/dev/null || echo "bun is required for fetch.ts fallback"

Install Bun dependencies for the bundled script:

cd ~/.claude/skills/web-fetch && bun install

Default Workflow

Use this as the default flow for any URL:

URL="<url>"
CONTENT_TYPE="$(curl -sIL "$URL" | awk -F': ' 'tolower($1)=="content-type"{print tolower($2)}' | tr -d '\r' | tail -1)"

if echo "$CONTENT_TYPE" | grep -q "markdown"; then
  curl -sL "$URL"
else
  curl -sL "$URL" \
    | html2markdown \
        --include-selector "article,main,[role=main]" \
        --exclude-selector "nav,header,footer,script,style"
fi