Claude MarketplacesSubmit
google avatar

Google Agents Cli Eval

google/agents-cliOfficial

This handles the evaluation workflow for Google's Agent Development Kit, covering everything from writing evalsets to debugging why your agent scores tanked. The real value is in the eval-fix loop guidance: it walks you through the 5-10+ iteration cycle you'll actually go through, with a useful table calling out shortcuts that waste time (like lowering thresholds instead of fixing your agent). You get concrete metric selection advice (tool_trajectory_avg_score for CI/CD, final_response_match_v2 for semantic checks), schema examples for both evalsets and config files, and separate references for user simulation, multimodal inputs, and built-in tools. Use this when you need to systematically improve agent quality rather than just eyeballing outputs.

skills.sh installs
558.9K
GitHub stars
6.3K
last updated
Aug 21, 2026
View on GitHub

Install

$ npx -y skills add google/agents-cli --skill google-agents-cli-eval --agent claude-code

Installs into .claude/skills of the current project.

Files

Agent Evaluation Guide

Requires: agents-cli (uv tool install google-agents-cli) — install uv first if needed.

Scaffolded project? If you used /google-agents-cli-scaffold, dataset and a custom metric are already scaffolded in tests/eval/ (Python projects) or eval/ (Go projects). For simplicity, this skill and its references use the Python directory layout; adjust accordingly if you've scaffolded a Go agent. You already have agents-cli eval run (chains generate + grade), tests/eval/datasets/, and tests/eval/eval_config.yaml. Start with executing eval run and iterate from there.

Reference Files

SKILL.md on GitHub

Sponsors

Promote your product hereReach 125k+ visitors monthlyPromote your product hereReach 125k+ visitors monthlyAgent CaffeineSponsoredClose your lid, come back to finished workSnugSponsoredThe clipboard manager for the AI eraPromote your product hereReach 125k+ visitors monthlyWaxSponsoredBlock anything with supervision, without erasing your iPhonePromote your product hereReach 125k+ visitors monthlyknobsSponsoredSee your site on any phone, any day, in any language

Details

Repository
github.com/google/agents-cli
Categories
Backend & APIs, DevOps & CI/CD, Code Review & Quality, AI & Agent Building, Data Science & ML, Release Management, Automation & Workflows, CLI & Terminal
First seen
May 16, 2026

More from google/agents-cli

All 7

Recommended

More Backend & APIs
1google avatargoogle-agents-cli-observabilitygoogle/agents-cliThis skill should be used when the user wants to "set up tracing", "monitor my ADK agent", "configure logging", "add observability", "debug production traffic",558.7Kskills.sh installs
2microsoft avatarazure-upgrademicrosoft/azure-skillsAssess and upgrade Azure workloads between plans, tiers, or SKUs, or modernize Azure SDK dependencies in source code. WHEN: upgrade Consumption to Flex Consumpt555.8Kskills.sh installs
3xixu-me avataruse-my-browserxixu-me/skillsUse when work depends on the user's live browser session or visible rendered state rather than static fetches, especially for browser debugging contexts or DevT519Kskills.sh installs
4xixu-me avatarxgetxixu-me/skillsUse when tasks involve Xget URL rewriting, registry/package/container/API acceleration, integrating Xget into Git, download tools, package managers, container b517.1Kskills.sh installs
5larksuite avatarlark-maillarksuite/cli飞书邮箱：Use when user mentions 起草邮件、写邮件、草稿、发送/回复/转发邮件、查阅邮件、看邮件、搜索邮件、邮件文件夹、邮件标签、邮件联系人、监听新邮件、邮件收信规则等；use for mail/email intent only. Do not use for docs/sheets/calen479.1Kskills.sh installs
6nextlevelbuilder avatarui-ux-pro-maxnextlevelbuilder/ui-ux-pro-max-skillUI/UX design intelligence for web and mobile. Searchable local database with 84 styles, 192 color palettes, 74 font pairings, 192 product types, 98 UX guideline443.3Kskills.sh installs