Url

This does exactly what you'd expect: pulls down a webpage with httpx, runs it through BeautifulSoup to strip out nav bars, scripts, and style tags, then hands you the clean text. It auto-triggers when you paste a URL or mention fetching a webpage. The AGPL license means if you modify and distribute it, you'll need to share your changes. Honestly, this is the kind of utility that should probably just be built in, but until then it's a straightforward 30-line solution to a common need. No frills, no configuration, just text extraction.

GitHub stars: 452
last updated: Jun 11, 2026

View on GitHub

Install

$ npx -y skills add axoviq-ai/synthadoc --skill url --agent claude-code

Installs into .claude/skills of the current project.

Files

URL Skill

Fetches a web URL using httpx, strips navigation/script/style tags with BeautifulSoup, and returns clean body text. PDF URLs are extracted with pypdf (primary) and pdfminer.six (fallback).

Setup

pip install httpx beautifulsoup4

# Optional — needed only if you ingest PDF URLs:
pip install pypdf pdfminer.six

Standalone usage

import asyncio
from synthadoc.skills.url.scripts.main import UrlSkill

skill = UrlSkill()

async def main():
    result = await skill.extract("https://example.com/article")
    print(result.text)          # clean body text
    print(result.metadata)      # {"url": "https://..."}

asyncio.run(main())

DomainBlockedException is raised when the site returns HTTP 401, 403, or 429. Catch it to log and skip the domain:

from synthadoc.skills.base import DomainBlockedException

try:
    result = await skill.extract(url)
except DomainBlockedException as e:
    print(f"Blocked: {e.domain} (HTTP {e.status_code})")