Create Task

Walks you through building a Harbor task from scratch, which means creating the instruction prompt, setting up the Docker environment, writing a verifier (usually the hardest part), and scripting a reference solution. The guidance is opinionated about verifier choice: it nudges you toward Reward Kit for multi-criteria grading, pytest for straightforward assertions, or plain shell for single-command checks. Useful if you're designing agent benchmarks or evals and need help structuring the pieces, especially around network policies, timeout configs, and the difference between shared versus separate verifier environments. The step-by-step format keeps you from dumping half-baked scaffolding and forgetting to test the Dockerfile interactively.

skills.sh installs: 920
GitHub stars: 6.9K
last updated: Aug 21, 2026

View on GitHub

Install

$ npx -y skills add harbor-framework/harbor --skill create-task --agent claude-code

Installs into .claude/skills of the current project.

Files

Guide the user through creating a new Harbor task end-to-end. Don't just dump commands — walk them through each decision, especially around the verifier (which is usually the hardest part).

Step 1: Scaffold the task

harbor task init "<org>/<task-name>"

Useful flags:

--description "..."
--author "Jane Doe <jane@example.com>" (repeat for multiple authors)
--no-pytest — skip the pytest test template (use if planning Reward Kit or custom verifier)
--no-solution — skip solution/ directory
--metadata-template path.toml — pre-populate task.toml

Produces:

<task-name>/
├── instruction.md         # Task prompt for the agent
├── task.toml              # Config and metadata
├── environment/Dockerfile # Container definition
├── solution/solve.sh      # Reference solution (optional)
└── tests/test.sh          # Verifier script

If the user wants a multi-step task (ordered steps with per-step instructions, tests, and early stopping against a shared container), scaffold the single-step layout first, then convert to the steps/ layout described in the Multi-step tasks section below.