Arize Evaluator

This handles the full lifecycle of LLM-as-judge and code-based evaluators in Arize: creating evaluators with custom prompts and classification choices, mapping template variables to actual span or experiment columns, running evals against traces or datasets, and setting up continuous monitoring as new data arrives. You can score at span, trace, or session granularity, which matters if you're evaluating multi-turn conversations or agent trajectories instead of single outputs. The skill wraps the ax CLI and assumes you've already got Arize credentials configured. One thing to watch: if an eval run fails, it won't fake results or give you manual scores. It tells you what broke and points you at the integration settings or support.

skills.sh installs: 3.2K
GitHub stars: 72
last updated: Aug 21, 2026

View on GitHub

Install

$ npx -y skills add arize-ai/arize-skills --skill arize-evaluator --agent claude-code

Installs into .claude/skills of the current project.

Files

Arize Evaluator Skill

SPACE — --space flags accept a space name (e.g., my-workspace) or a base64 space ID (e.g., U3BhY2U6...). Find yours with ax spaces list.

This skill covers designing, creating, and running evaluators on Arize — both LLM-as-judge (template) evaluators and code evaluators (deterministic, no LLM required). An evaluator defines the judge; a task is how you run it against real data.