Arize Evaluator

This is a comprehensive workflow tool for running LLM-as-judge evaluations through Arize's platform. It handles the full lifecycle: creating evaluators with custom judge prompts and classification choices, mapping template variables to your actual span or experiment data, running evals at span/trace/session granularity, and setting up continuous monitoring for production traffic. The skill knows how to troubleshoot common issues like missing credentials or failed API calls, and it includes a strict rule against fabricating evaluation results if something goes wrong. Most useful when you need to systematically score things like hallucination, faithfulness, or relevance across LLM outputs at scale. The documentation is quite detailed on column mapping and the differences between evaluating individual spans versus entire conversation sessions.

skills.sh installs: 1.1K
GitHub stars: 41K
last updated: Aug 21, 2026

View on GitHub

Install

$ npx -y skills add github/awesome-copilot --skill arize-evaluator --agent claude-code

Installs into .claude/skills of the current project.

Files

Arize Evaluator Skill

SPACE — All --space flags and the ARIZE_SPACE env var accept a space name (e.g., my-workspace) or a base64 space ID (e.g., U3BhY2U6...). Find yours with ax spaces list.

This skill covers designing, creating, and running LLM-as-judge evaluators on Arize. An evaluator defines the judge; a task is how you run it against real data.