Agent Evaluation

This is the framework you need when your agent works great in demos but falls apart in production. It gives you statistical evaluation patterns that run tests multiple times to catch stochastic behavior, plus behavioral contract testing to enforce hard boundaries on what agents can and cannot do. The statistical evaluator calculates pass rates with confidence intervals, tracks behavior consistency across runs, and flags concerns like high variance or unstable outputs. The behavioral contract pattern is especially useful for production agents where you need guarantees about tone, scope, or safety. Remember that even top agents score under 50% on real world benchmarks, so this focuses on the metrics that actually matter for reliability.

skills.sh installs: 815
GitHub stars: 43.1K
last updated: Jul 14, 2026

View on GitHub

Install

$ npx -y skills add sickn33/antigravity-awesome-skills --skill agent-evaluation --agent claude-code

Installs into .claude/skills of the current project.

Files

Agent Evaluation

Evaluate observable agent behavior against task-specific cases. Modified by AAS maintainers on 2026-09-05 to remove unsupported benchmark claims, correct uncertainty/error reporting and separate optional architecture sketches from the operating procedure.

When to Use

Use when comparing a changed agent, prompt or tool configuration, reproducing an observed failure, or estimating reliability on a declared task distribution. Do not infer product readiness from a public benchmark percentage or a generic score threshold.

Prerequisites

A versioned case set with expected observable outcomes and permission boundaries.
A known baseline and candidate revision, including model, prompt, tools, configuration and runtime versions.
Authorized synthetic or redacted inputs, isolated targets and a bounded token, time and cost budget.
A verifier that distinguishes wrong outcomes, expected safe rejections, evaluator failures and infrastructure outages. Provider access is needed only if the declared evaluation calls that provider.