Ray Train

If you're scaling PyTorch or TensorFlow training beyond a single machine, this handles the distributed orchestration without rewriting your code. Wrap your training function, specify worker count, and Ray manages GPU allocation, fault tolerance, and metric aggregation across nodes. The real win is tight integration with Ray Tune for distributed hyperparameter sweeps. Setup is genuinely simpler than raw PyTorch DDP or Horovod, though you're buying into the Ray ecosystem. Works well for large model training or running dozens of experiments in parallel across a cluster.

skills.sh installs: 1.1K
GitHub stars: 14.4K
last updated: Aug 21, 2026

View on GitHub

Install

$ npx -y skills add orchestra-research/ai-research-skills --skill ray-train --agent claude-code

Installs into .claude/skills of the current project.

Files

Ray Train - Distributed Training Orchestration

Quick start

Ray Train scales machine learning training from single GPU to multi-node clusters with minimal code changes.

Installation:

pip install -U "ray[train]"

Basic PyTorch training (single node):

import ray
from ray import train
from ray.train import ScalingConfig
from ray.train.torch import TorchTrainer
import torch
import torch.nn as nn

# Define training function
def train_func(config):
    # Your normal PyTorch code
    model = nn.Linear(10, 1)
    optimizer = torch.optim.SGD(model.parameters(), lr=0.01)

    # Prepare for distributed (Ray handles device placement)
    model = train.torch.prepare_model(model)

    for epoch in range(10):
        # Your training loop
        output = model(torch.randn(32, 10))
        loss = output.sum()
        loss.backward()
        optimizer.step()
        optimizer.zero_grad()

        # Report metrics (logged automatically)
        train.report({"loss": loss.item(), "epoch": epoch})

SKILL.md on GitHub

Details

Repository: github.com/orchestra-research/ai-research-skills
Categories: AI & Agent Building, Data Science & ML, Automation & Workflows
First seen: Jun 3, 2026