Skill Creator

This is meta tooling for the skill system itself. It walks you through the full cycle of creating a new skill from scratch, running evals against test prompts, benchmarking performance with quantitative metrics, and iterating based on results. It also optimizes skill descriptions for better triggering accuracy, which matters because Claude tends to undertrigger skills. The workflow is flexible enough to handle users who want rigorous testing or just want to vibe check their changes. If you're building skills or wondering why your skill isn't firing when it should, this handles both problems.

GitHub stars: 722
last updated: Jun 1, 2026

View on GitHub

Install

$ npx -y skills add countbot-ai/countbot --skill skill-creator --agent claude-code

Installs into .claude/skills of the current project.

Files

Skill Creator

A skill for creating new skills and iteratively improving them.

At a high level, the process of creating a skill goes like this:

Decide what you want the skill to do and roughly how it should do it
Write a draft of the skill
Create a few test prompts and run claude-with-access-to-the-skill on them
Help the user evaluate the results both qualitatively and quantitatively
- While the runs happen in the background, draft some quantitative evals if there aren't any (if there are some, you can either use as is or modify if you feel something needs to change about them). Then explain them to the user (or if they already existed, explain the ones that already exist)
- Use the eval-viewer/generate_review.py script to show the user the results for them to look at, and also let them look at the quantitative metrics
Rewrite the skill based on feedback from the user's evaluation of the results (and also if there are any glaring flaws that become apparent from the quantitative benchmarks)
Repeat until you're satisfied
Expand the test set and try again at larger scale