
Content Extract
This is a smart URL-to-Markdown converter that knows when to take shortcuts and when to bring out the heavy machinery. It tries a cheap web_fetch first, and if that hits anti-scraping walls or returns garbage (common with WeChat articles on mp.weixin.qq.com), it falls back to the MinerU API for high-fidelity extraction. The output is a structured JSON contract with source URLs, the cleaned Markdown, and file paths so you can always trace where content came from. Built for workflows where you need reliable extraction without babysitting failed requests or dealing with "please open in WeChat client" dead ends. The domain whitelist for known problem sites is a nice touch.
- skills.sh installs
- 161
- GitHub stars
- 436
- last updated
- Aug 21, 2026
Install
$ npx -y skills add blessonism/openclaw-search-skills --skill content-extract --agent claude-codeInstalls into .claude/skills of the current project.
Files
content-extract — 上层内容解析入口(MCP 语义对齐,但不跑 MCP Server)
目标:把“给我一个 URL → 产出可读 Markdown + 可追溯入口”变成一个统一入口,供后续所有业务 skill(github-explorer、写作类 skills、日报等)复用。
核心原则(来自你发的 Excel Skill 拆解文章的启发):
- 行为规约层:永远给出可追溯入口(原文 URL + 解析产物路径/链接),绝不编造来源。
- Token 探针:先用低成本 probe 判断可不可以直接抓;不行再走重解析(MinerU)。
- 反弹机制:失败时返回“下一步动作建议”,而不是一堆异常栈。
工作流(Decision Tree)
输入:url
- Domain Whitelist(跳过 probe):若 URL 属于高概率反爬/动态站点(微信/知乎等),直接走 MinerU
- 白名单文件:
references/domain-whitelist.md - 对命中白名单的 URL:强制
model_version=MinerU-HTML
- Probe(低成本):优先用
web_fetch(url)
- 目标:拿到正文 markdown(便宜、快)
- 判断“失败/不合格”条件(见
references/heuristics.md)包括:- 403/401/反爬
- 只有“环境异常/验证码/请在微信打开”等提示
- 内容极短/明显导航页/丢正文
- Fallback(高保真):走 MinerU 官方 API
- 调用下游 driver:
skills/mineru-extract/scripts/mineru_parse_documents.py - 对 HTML 页面(微信等):强制
model_version=MinerU-HTML
- 输出统一结果合同(Result Contract)
无论用 probe 还是 MinerU,都返回同一套结构:
Sponsors
Details
- Repository
- github.com/blessonism/openclaw-search-skills
- Categories
- Backend & APIs, Automation & Workflows
- First seen
- Jun 3, 2026