Hive Create Task
This is a wizard that walks you through creating a complete hive task from scratch. It asks questions to nail down the problem, helps you design the eval (which it emphasizes as the most important part), defines what agents can and can't touch, then scaffolds the whole repo with program.md, eval.sh, requirements, and a baseline artifact. The real value is in phase five, where it actually runs the eval and loops until the baseline works and scores in a reasonable range. If your baseline scores zero or near perfect, it catches that and helps you tune difficulty. Saves you from publishing broken tasks or ones with no headroom for agents to improve.
- skills.sh installs
- 533
- GitHub stars
- 219
- last updated
- Aug 21, 2026
Install
$ npx -y skills add rllm-org/hive --skill hive-create-task --agent claude-codeInstalls into .claude/skills of the current project.
Files
Hive Create Task
Interactive wizard for designing and creating a new hive task. Guide the user through each phase with clarifying questions. The goal is to produce a complete, tested task repo that agents can immediately clone and work on.
Principle: Ask the right questions to help the user clarify their thinking. A good task needs a good eval — spend most of the effort there. Don't move on until the user is satisfied with each phase.
UX Note: Use AskUserQuestion for all user-facing questions.
Task Repo Structure
Required files
| File | Purpose |
|---|---|
program.md | Instructions for the agent: what to modify, how to eval, the experiment loop, and constraints |
eval/eval.sh | Evaluation script — must be runnable via bash eval/eval.sh and print a score |
requirements.txt | Python dependencies |
README.md | Short description, quickstart, and leaderboard link |
Recommended files
| File | Purpose |
|---|---|
prepare.sh | Setup script — downloads data, installs deps. Recommended but not required. |
The artifact (free-form)
The rest depends on the task type — this is what agents evolve:
Sponsors
Agent CaffeineSponsoredClose your lid, come back to finished work
SnugSponsoredThe clipboard manager for the AI eraPromote your product hereReach 125k+ visitors monthly
WaxSponsoredBlock anything with supervision, without erasing your iPhonePromote your product hereReach 125k+ visitors monthly
knobsSponsoredSee your site on any phone, any day, in any languageDetails
- Repository
- github.com/rllm-org/hive
- Categories
- Testing & QA, Release Management, Design & UI/UX
- First seen
- Jun 3, 2026
More from rllm-org/hive
All 3Recommended
More Testing & QA
rebuttalwanshuiyin/auto-claude-code-research-in-sleepWorkflow 4: Submission rebuttal pipeline. Parses external reviews, enforces coverage and grounding, drafts a safe text-only rebuttal under venue limits, and man532skills.sh installs
cli-anything-iterm2hkuds/cli-anythingProvides the cli-anything-iterm2 commands — the only way to actually send text to iTerm2 sessions, read live terminal output and scrollback history, manage wind530skills.sh installs
search-consoleopenclaudia/openclaudia-skillsPull Google Search Console data and perform search performance analysis. Use when asked about search rankings, clicks, impressions, CTR, index coverage, Core We530skills.sh installs
prototypeplugin87/ux-ui-agent-skillsMove an idea up the fidelity ladder (content-first → wireframe → low-fi → high-fi → code) with a validation plan at each level, plus user-journey mapping and us528skills.sh installs
longbridge-consensuslongbridge/skillsAnalyst consensus snapshot for listed companies via Longbridge — current revenue / EPS / target-price consensus estimates and analyst rating distribution. For r524skills.sh installslongbridge-sector-monitorlongbridge/skillsSector rotation continuous monitoring via Longbridge — 6–12 month trend tracking across economic cycles (recovery / overheating / stagflation / recession), iden521skills.sh installs