Claude MarketplacesSubmit
nanmicoder avatar

China News Crawler

nanmicoder/newscrawler

A straightforward news scraper that pulls article content from 12 platforms including WeChat Official Accounts, Toutiao, BBC, CNN, and Twitter, then outputs both JSON and Markdown. It auto-detects the platform from the URL and handles the HTTP juggling with curl_cffi for browser impersonation. The Twitter support is nice because it works with public tweets out of the box and accepts cookies for protected ones. Built with uv for dependency management and includes all the crawler code in one package, so you can lift it into another project without hunting down pieces. Good for when you need clean article extraction without building parsers for each platform yourself.

skills.sh installs
118
GitHub stars
639
last updated
Aug 21, 2026
View on GitHub

Install

$ npx -y skills add nanmicoder/newscrawler --skill china-news-crawler --agent claude-code

Installs into .claude/skills of the current project.

Files

A straightforward news scraper that pulls article content from 12 platforms including WeChat Official Accounts, Toutiao, BBC, CNN, and Twitter, then outputs both JSON and Markdown. It auto-detects the platform from the URL and handles the HTTP juggling with curl_cffi for browser impersonation. The Twitter support is nice because it works with public tweets out of the box and accepts cookies for protected ones. Built with uv for dependency management and includes all the crawler code in one package, so you can lift it into another project without hunting down pieces. Good for when you need clean article extraction without building parsers for each platform yourself.

Browse the repository on GitHub

Sponsors

Promote your product hereReach 125k+ visitors monthlyPromote your product hereReach 125k+ visitors monthlyAgent CaffeineSponsoredClose your lid, come back to finished workSnugSponsoredThe clipboard manager for the AI eraPromote your product hereReach 125k+ visitors monthlyWaxSponsoredBlock anything with supervision, without erasing your iPhonePromote your product hereReach 125k+ visitors monthlyknobsSponsoredSee your site on any phone, any day, in any language

Details

Repository
github.com/nanmicoder/newscrawler
First seen
Jun 3, 2026

More from nanmicoder/newscrawler

All 2

Recommended

1juliusbrussee avatarcavemanjuliusbrussee/cavemanUltra-compressed communication mode. Cuts output tokens 65% (measured) by speaking like caveman while keeping full technical accuracy. Supports intensity levels618.3Kskills.sh installs
2mattpocock avatargrill-memattpocock/skillsA relentless interview to sharpen a plan or design.1.5Mskills.sh installs
3shadcn avatarimproveshadcn/improveSurvey any codebase as a senior advisor and produce prioritized, self-contained implementation plans for OTHER models/agents to execute. Strictly read-only on s40Kskills.sh installs
4obra avatarsystematic-debuggingobra/superpowersUse when encountering any bug, test failure, or unexpected behavior, before proposing fixes326.7Kskills.sh installs
5forrestchang avatarkarpathy-guidelinesforrestchang/andrej-karpathy-skillsBehavioral guidelines to reduce common LLM coding mistakes through explicit assumptions, simplicity, and verifiable success criteria.17.9Kskills.sh installs
6vercel-labs avatarfind-skillsvercel-labs/skillsHelps users discover and install agent skills when they ask questions like "how do I do X", "find a skill for X", "is there a skill that can...", or express int4.3Mskills.sh installs