Builds structured A/B test plans for paid ads with proper statistical rigor. Takes your test idea and walks you through hypothesis framing, sample size calculations, duration estimates, and platform-specific setup for Meta, Google, LinkedIn, and TikTok. The hypothesis framework alone is worth it: forces you to state the variable, expected impact size, and reasoning before you spend a dollar. Includes lookup tables for required sample sizes at different conversion rates and minimum detectable effects, which beats guessing when to call a test. Honestly most useful for preventing the classic mistakes like ending tests early or changing multiple variables at once. If you run paid experiments beyond just launching stuff and hoping, this gives you the structure to do it properly.

skills.sh installs
3.9K
GitHub stars
10.9K
last updated
Aug 21, 2026

Install

$ npx -y skills add agricidaniel/claude-ads --skill ads-test --agent claude-code

Installs into .claude/skills of the current project.

Files

Paid Media Experiment

  1. State the decision, causal hypothesis, treatment, control, randomization unit, population, primary metric, guardrails, minimum effect, and stopping rule.
  2. Check platform constraints, overlapping experiments, conversion lag, seasonality, interference, and measurement quality.
  3. Calculate sample and duration from declared assumptions; disclose approximations.
  4. Change one decision surface unless the design explicitly estimates interactions.
  5. Pre-register exclusions, quality checks, analysis, and decision thresholds.
  6. For readout, verify assignment integrity and data completeness before estimating effect and uncertainty.
  7. Return setup or readout in versioned JSON with a plain-language decision.

Do not repeatedly peek and stop on a favorable result, call underpowered noise a winner, or generalize beyond the tested population.

Sponsors

Details

Repository
github.com/agricidaniel/claude-ads
Categories
Testing & QA, Data Science & ML, Productivity & Planning, Design & UI/UX
First seen
Jun 3, 2026

More from agricidaniel/claude-ads

All 23
1microsoft avatarplaywright-devmicrosoft/playwrightExplains how to develop Playwright - add APIs, MCP tools, CLI commands, and vendor dependencies.3.8Kskills.sh installs
2pytorch avatarpyrefly-type-coveragepytorch/pytorchMigrate a file to use stricter Pyrefly type checking with annotations required for all functions, classes, and attributes.3.2Kskills.sh installs
3phuryn avatarab-test-analysisphuryn/pm-skillsAnalyze A/B test results with statistical significance, sample size validation, confidence intervals, and ship/extend/stop recommendations. Use when evaluating 3Kskills.sh installs
4eronred avatarab-test-store-listingeronred/aso-skillsWhen the user wants to A/B test App Store product page elements to improve conversion rate. Also use when the user mentions "A/B test", "product page optimizati2.5Kskills.sh installs
5owl-listener avatardesign-qa-checklistowl-listener/designer-skillsCreate QA checklists for verifying design implementation accuracy.2.5Kskills.sh installs
6owl-listener avatartest-scenarioowl-listener/designer-skillsGenerates structured usability test scenarios with realistic tasks, success criteria, and facilitation notes — ready to run with real participants or in a moder2.3Kskills.sh installs