Eval Harness

This is a structured framework for writing and running evals before you code, treating them like unit tests for AI development. You define pass/fail criteria upfront, implement against those specs, then measure reliability with pass@k metrics (like pass@3 > 90% for new features). It supports code-based graders for deterministic checks, model graders for subjective evaluation, and human review flags for security-critical changes. The workflow is simple: define evals in .claude/evals/, run them during development, track regressions. Honestly, this feels most valuable if you're already bought into test-driven development and want that discipline for prompt engineering. If you're just prototyping or doing one-off tasks, the overhead might not pay off.

skills.sh installs: 5.7K
GitHub stars: 300.3K
last updated: Aug 21, 2026

View on GitHub

Install

$ npx -y skills add affaan-m/ecc --skill eval-harness --agent claude-code

Installs into .claude/skills of the current project.

Files

Eval Harness Skill

A formal evaluation framework for Claude Code sessions, implementing eval-driven development (EDD) principles.

When to Activate

Setting up eval-driven development (EDD) for AI-assisted workflows
Defining pass/fail criteria for Claude Code task completion
Measuring agent reliability with pass@k metrics
Creating regression test suites for prompt or agent changes
Benchmarking agent performance across model versions

Philosophy

Eval-Driven Development treats evals as the "unit tests of AI development":

Define expected behavior BEFORE implementation
Run evals continuously during development
Track regressions with each change
Use pass@k metrics for reliability measurement

Eval Types

Capability Evals

Test if Claude can do something it couldn't before:

[CAPABILITY EVAL: feature-name]
Task: Description of what Claude should accomplish
Success Criteria:
  - [ ] Criterion 1
  - [ ] Criterion 2
  - [ ] Criterion 3
Expected Output: Description of expected result

SKILL.md on GitHub