Agent Eval
This is a benchmarking harness for comparing coding agents on your actual codebase instead of relying on vibes. You define tasks as YAML files with prompts and pass/fail criteria (pytest, grep patterns, or LLM judges), then run multiple agents in isolated git worktrees and collect pass rates, cost, and timing. The real value is reproducibility: pinned commits, multiple trials to capture variance, and structured task definitions you can version control. Best for teams evaluating which agent to adopt or checking for regressions after model updates. The metrics table is straightforward, three runs minimum recommended since agents are non-deterministic.
- skills.sh installs
- 4.5K
- GitHub stars
- 229.1K
- last updated
- Jul 14, 2026
Install
$ npx -y skills add affaan-m/everything-claude-code --skill agent-eval --agent claude-codeInstalls into .claude/skills of the current project.
Files
Agent Eval Skill
A lightweight CLI tool for comparing coding agents head-to-head on reproducible tasks. Every "which coding agent is best?" comparison runs on vibes — this tool systematizes it.
When to Activate
- Comparing coding agents (Claude Code, Aider, Codex, etc.) on your own codebase
- Measuring agent performance before adopting a new tool or model
- Running regression checks when an agent updates its model or tooling
- Producing data-backed agent selection decisions for a team
Installation
Note: Install agent-eval from its repository after reviewing the source.
Core Concepts
YAML Task Definitions
Define tasks declaratively. Each task specifies what to do, which files to touch, and how to judge success:
Sponsors
Agent CaffeineSponsoredClose your lid, come back to finished work
SnugSponsoredThe clipboard manager for the AI eraPromote your product hereReach 125k+ visitors monthly
WaxSponsoredBlock anything with supervision, without erasing your iPhonePromote your product hereReach 125k+ visitors monthly
knobsSponsoredSee your site on any phone, any day, in any languageDetails
- Repository
- github.com/affaan-m/everything-claude-code
- Category
- AI & Agent Building
- First seen
- Apr 16, 2026
More from affaan-m/everything-claude-code
All 256- Claude Devfleet4.5K
- Dmux Workflows4.5K
- Perl Patterns4.5K
- Perl Security4.5K
- Perl Testing4.5K
- Returns Reverse Logistics4.5K
- Team Builder4.5K
- X Api4.5K
- Design System4.4K
- Nuxt4 Patterns4.4K
- Browser Qa4.3K
- Jira Integration4.3K
- Rules Distill4.3K
- Click Path Audit4.2K
- Git Workflow4.2K
- Hexagonal Architecture4.2K
- Remotion Video Creation4.2K
- Santa Method4.2K
- Skill Comply4.2K
- Laravel Verification4.1K
- Nestjs Patterns4.1K
- Autonomous Agent Harness4K
- Benchmark4K
- Brand Voice4K
Recommended
More AI & Agent Building
dependabotgithub/awesome-copilotComprehensive guide for configuring and managing GitHub Dependabot. Use this skill when users ask about creating or optimizing dependabot.yml files, managing De4.4Kskills.sh installs
agent-md-refactorsoftaworks/agent-toolkitRefactor bloated AGENTS.md, CLAUDE.md, or similar agent instruction files to follow progressive disclosure principles. Splits monolithic files into organized, l4.3Kskills.sh installs
autonomous-agent-harnessaffaan-m/everything-claude-codeautonomous agent harness4Kskills.sh installs
create-custom-agentdotnet/skillsCreates VS Code custom agent files (.agent.md) for specialized AI personas with tools, instructions, and handoffs. Use when scaffolding new custom agents, confi4Kskills.sh installs
mcp-deepwikiaahl/skillsSkills for accessing and searching docs in DeepWiki/GitHub’s public code repositories can help users understand open-source project source codes, and users can 3Kskills.sh installs