Ml Training Recipes

This is a comprehensive PyTorch training reference that covers everything from optimizer selection (Muon vs AdamW for different parameter types) to domain-specific patterns for LLMs, vision, diffusion, and biomedical applications. It includes actual code snippets for training loops, learning rate schedules, and mixed precision setup, plus decision tables for architecture selection based on data scale. The scaling laws section gives you Chinchilla-optimal token counts, and there's practical debugging advice for loss spikes and OOM errors. What makes this useful is the specificity: it tells you to use lr * (d_model / 768)^(-0.5) for dimension scaling and eps=1e-10 for AdamW in bfloat16, not just "tune your hyperparameters." If you're setting up training from scratch or debugging why your loss won't converge, this beats hunting through papers and GitHub issues.

skills.sh installs: 996
GitHub stars: 14.4K
last updated: Aug 21, 2026

View on GitHub

Install

$ npx -y skills add orchestra-research/ai-research-skills --skill ml-training-recipes --agent claude-code

Installs into .claude/skills of the current project.

Files

ML Training Recipes

Battle-tested patterns for PyTorch training across domains. Drawn from production codebases (Karpathy's autoresearch/nanochat, torchvision, HuggingFace) and modern training practice.

Reference files (read when needed)

references/architecture.md — Transformer/LLM architecture code patterns, weight init
references/optimizers.md — Muon, AdamW hybrid, per-group LR, compiled optimizer steps
references/domain-specific.md — Vision, diffusion, contrastive, distributed, checkpointing, data loading
references/scaling-and-selection.md — Scaling laws, compute budget tables, decision trees, DGX Spark
references/biomedical.md — Drug discovery, protein models, medical imaging, genomics, clinical NLP
references/experiment-loop.md — Autonomous experiment loop (autoresearch keep/discard/revert)