Experiment Design

Structures your ML experiments into four progressive stages borrowed from AI-Scientist-v2: get something working, tune the baseline, try creative ideas, then run ablations. It generates experiment plans with baselines, datasets, hyperparameter grids, and metrics in a JSON format you can actually execute against. The staged approach is smart because it stops you from tweaking architectures before you have stable training curves, which is how most research projects waste their first two weeks. Includes a Python script that outputs either JSON or markdown, and it enforces multi-seed runs for significance testing. Honestly most useful as a planning framework to keep experiments systematic rather than as automation, since research rarely follows a script perfectly.

skills.sh installs: 1.8K
GitHub stars: 459
last updated: Aug 21, 2026

View on GitHub

Install

$ npx -y skills add lingzhi227/agent-research-skills --skill experiment-design --agent claude-code

Installs into .claude/skills of the current project.

Files

Experiment Design

Design structured, progressive experiment plans for research papers.

Input

$0 — Research idea, plan, or method description

References

4-stage progressive experiment prompts: ~/.claude/skills/experiment-design/references/stage-prompts.md

Scripts

Generate experiment design

python ~/.claude/skills/experiment-design/scripts/design_experiments.py --plan research_plan.json --output experiment_design.json
python ~/.claude/skills/experiment-design/scripts/design_experiments.py --method "contrastive learning" --task classification --format markdown

Generates baselines, ablation matrix, hyperparameter grid, metric selection. Stdlib-only.