Sentencepiece
This is a tokenization tool that trains on raw text without needing language-specific preprocessing, which makes it genuinely useful for multilingual work. The performance numbers are solid: 50k sentences per second with only 6MB memory footprint. You'd reach for this when building models that need to handle multiple languages consistently, especially CJK text where word boundaries aren't obvious. The deterministic vocabulary is a nice bonus for reproducibility. It's been installed 281 times and passes most security audits, though a couple show warnings worth checking if you're in a strict environment.
- skills.sh installs
- 277
- GitHub stars
- 33.2K
- last updated
- Aug 21, 2026
Install
$ npx -y skills add davila7/claude-code-templates --skill sentencepiece --agent claude-codeInstalls into .claude/skills of the current project.
Files
This is a tokenization tool that trains on raw text without needing language-specific preprocessing, which makes it genuinely useful for multilingual work. The performance numbers are solid: 50k sentences per second with only 6MB memory footprint. You'd reach for this when building models that need to handle multiple languages consistently, especially CJK text where word boundaries aren't obvious. The deterministic vocabulary is a nice bonus for reproducibility. It's been installed 281 times and passes most security audits, though a couple show warnings worth checking if you're in a strict environment.
Browse the repository on GitHubSponsors
Agent CaffeineSponsoredClose your lid, come back to finished work
SnugSponsoredThe clipboard manager for the AI eraPromote your product hereReach 125k+ visitors monthly
WaxSponsoredBlock anything with supervision, without erasing your iPhonePromote your product hereReach 125k+ visitors monthly
knobsSponsoredSee your site on any phone, any day, in any languageDetails
- Repository
- github.com/davila7/claude-code-templates
- First seen
- Jun 3, 2026
More from davila7/claude-code-templates
All 618- Using Git Worktrees277
- Free Tool Strategy276
- Qms Audit Expert276
- Railway Metrics276
- Nnsight Remote Interpretability275
- Pytdc275
- Railway Domain275
- Firecrawl Scraper274
- Simpo Training273
- Gene Database272
- Uniprot Database269
- File Uploads268
- Simpy265
- Meme Factory264
- Metabolomics Workbench Database264
- Slack Gif Creator264
- Concise Planning263
- Diffdock262
- Clinpgx Database261
- Tavily Web260
- Gget259
- Crafting Effective Readmes258
- Geo Fundamentals258
- String Database255
Recommended
cavemanjuliusbrussee/cavemanUltra-compressed communication mode. Cuts output tokens 65% (measured) by speaking like caveman while keeping full technical accuracy. Supports intensity levels618.3Kskills.sh installs
improveshadcn/improveSurvey any codebase as a senior advisor and produce prioritized, self-contained implementation plans for OTHER models/agents to execute. Strictly read-only on s40Kskills.sh installs
systematic-debuggingobra/superpowersUse when encountering any bug, test failure, or unexpected behavior, before proposing fixes326.7Kskills.sh installs
karpathy-guidelinesforrestchang/andrej-karpathy-skillsBehavioral guidelines to reduce common LLM coding mistakes through explicit assumptions, simplicity, and verifiable success criteria.17.9Kskills.sh installs
find-skillsvercel-labs/skillsHelps users discover and install agent skills when they ask questions like "how do I do X", "find a skill for X", "is there a skill that can...", or express int4.3Mskills.sh installs