Arize Experiment

This gives Claude direct control over Arize's experiment workflow through the ax CLI. It can create experiments against your datasets, export runs for analysis, and compare model outputs with evaluations like correctness or relevance. The skill enforces real API calls for every dataset example, no fake outputs or scores, which matters when you're actually benchmarking models. It handles both REST and Arrow Flight exports, automatically escalating to Flight when you hit the 500-run pagination limit. Good for A/B testing different prompts or models, running evals at scale, or pulling experiment data into your own analysis pipeline. Assumes you already have ax installed and an Arize profile configured.

skills.sh installs: 3.2K
GitHub stars: 72
last updated: Aug 21, 2026

View on GitHub

Install

$ npx -y skills add arize-ai/arize-skills --skill arize-experiment --agent claude-code

Installs into .claude/skills of the current project.

Files

Arize Experiment Skill

SPACE — --space flags accept a space name (e.g., my-workspace) or a base64 space ID (e.g., U3BhY2U6...). Find yours with ax spaces list.

Concepts

Experiment = a named evaluation run against a specific dataset version, containing one run per example
Experiment Run = the result of processing one dataset example -- includes the model output, optional evaluations, and optional metadata
Dataset = a versioned collection of examples; every experiment is tied to a dataset and a specific dataset version
Evaluation = a named metric attached to a run (e.g., correctness, relevance), with optional label, score, and explanation

The typical flow: export a dataset → process each example → collect outputs and evaluations → create an experiment with the runs.