Ab Testing

This helps you design and run A/B tests that actually produce valid results instead of noise. It walks you through hypothesis formation, sample size calculations, variant design, and statistical analysis with specific frameworks like "Because X, we believe Y will cause Z" and ICE scoring for prioritization. Goes beyond single tests to cover building a continuous experimentation program with backlog management and playbook development. The emphasis on not peeking early and pre-committing to sample sizes is refreshingly direct about the discipline required. Includes sample size tables, metric selection guidance, and documentation templates. Works with product marketing context files if they exist to avoid asking redundant questions.

skills.sh installs: 85.2K
GitHub stars: 59.5K
last updated: Aug 21, 2026

View on GitHub

Install

$ npx -y skills add coreyhaines31/marketingskills --skill ab-testing --agent claude-code

Installs into .claude/skills of the current project.

Files

A/B Test Setup

You are an expert in experimentation and A/B testing. Your goal is to help design tests that produce statistically valid, actionable results.

Initial Assessment

Check for product marketing context first: If .agents/product-marketing.md exists (or .claude/product-marketing.md, or the legacy product-marketing-context.md filename, in older setups), read it before asking questions. Use that context and only ask for information not already covered or specific to this task.

Before designing a test, understand:

Test Context - What are you trying to improve? What change are you considering?
Current State - Baseline conversion rate? Current traffic volume?
Constraints - Technical complexity? Timeline? Tools available?