Arize Experiment

This skill wraps the `ax` CLI to help you create, run, and analyze model evaluation experiments in Arize. It handles the full workflow: export a dataset, process examples through your model, collect outputs and evaluations, then push the runs back as a named experiment. The commands cover listing experiments, exporting results (with automatic escalation to Arrow Flight for bulk transfers over 500 runs), and creating new experiments from JSON, CSV, or Parquet files. Useful when you're A/B testing models, measuring accuracy across prompts, or benchmarking different configurations. The skill enforces a strict rule: it will never fabricate model outputs or scores, so you need real API access to run experiments properly.

skills.sh installs: 1.1K
GitHub stars: 41K
last updated: Aug 21, 2026

View on GitHub

Install

$ npx -y skills add github/awesome-copilot --skill arize-experiment --agent claude-code

Installs into .claude/skills of the current project.

Files

Arize Experiment Skill

SPACE — All --space flags and the ARIZE_SPACE env var accept a space name (e.g., my-workspace) or a base64 space ID (e.g., U3BhY2U6...). Find yours with ax spaces list.

Concepts

Experiment = a named evaluation run against a specific dataset version, containing one run per example
Experiment Run = the result of processing one dataset example -- includes the model output, optional evaluations, and optional metadata
Dataset = a versioned collection of examples; every experiment is tied to a dataset and a specific dataset version
Evaluation = a named metric attached to a run (e.g., correctness, relevance), with optional label, score, and explanation

The typical flow: export a dataset → process each example → collect outputs and evaluations → create an experiment with the runs.