News Extractor

Pulls article content from five major Chinese news platforms: WeChat Official Accounts, Toutiao, NetEase, Sohu, and Tencent News. You throw a URL at it and get back structured JSON plus a clean Markdown file with the article text, images, and metadata. Built on curl_cffi for browser simulation and includes retry logic, which matters when scraping these sites. Uses uv for dependency management, so everything runs in isolation. The output format is consistent across platforms, making it easy to build workflows on top of it. Worth noting this is explicitly for education and research, not production scraping at scale. Works well when you need to extract a few articles for analysis or archival without manually copying content.

GitHub stars: 246
last updated: May 2, 2026

View on GitHub

Install

$ npx -y skills add nanmicoder/skills-agent-proto --skill news-extractor --agent claude-code

Installs into .claude/skills of the current project.

Files

News Extractor Skill

从主流新闻平台提取文章内容，输出 JSON 和 Markdown 格式。

支持平台

平台	ID	URL 示例
微信公众号	wechat	`https://mp.weixin.qq.com/s/xxxxx`
今日头条	toutiao	`https://www.toutiao.com/article/123456/`
网易新闻	netease	`https://www.163.com/news/article/ABC123.html`
搜狐新闻	sohu	`https://www.sohu.com/a/123456_789`
腾讯新闻	tencent	`https://news.qq.com/rain/a/20251016A07W8J00`

依赖安装

本 skill 使用 uv 管理依赖。首次使用前需要安装：

# 项目级使用（推荐）
cd .claude/skills/news-extractor
uv sync

# 或用户级使用
cd ~/.claude/skills/news-extractor
uv sync

说明: 此 skill 可放置在项目级 (.claude/skills/) 或用户级 (~/.claude/skills/) 目录。项目级便于团队共享，用户级便于跨项目复用。

重要: 所有脚本必须使用 uv run 执行，不要直接用 python 运行。uv run 会自动使用项目虚拟环境中的依赖。

依赖列表

包名	用途
pydantic	数据模型验证
requests	HTTP 请求
curl_cffi	浏览器模拟抓取
tenacity	重试机制
parsel	HTML/XPath 解析
demjson3	非标准 JSON 解析

使用方式

基本用法

# 提取新闻，自动检测平台，输出 JSON + Markdown
uv run .claude/skills/news-extractor/scripts/extract_news.py "URL"

# 指定输出目录
uv run .claude/skills/news-extractor/scripts/extract_news.py "URL" --output ./output

SKILL.md on GitHub