Torchforge Rl Training

Meta's torchforge separates RL infrastructure from algorithms so you can experiment with GRPO, DAPO, or custom losses without rebuilding distributed plumbing. It wires together TorchTitan for training, vLLM for inference, and Monarch for actor coordination, letting you define reward functions and loss logic while it handles the weight syncing and GPU orchestration. You need at least 3 GPUs to run GRPO training, and the library is explicitly experimental with changing APIs. If you're prototyping agentic RL in PyTorch and want clean abstractions over the usual Ray-based stacks, this is worth trying. For production stability, Meta suggests looking at miles or verl instead.

skills.sh installs: 1K
GitHub stars: 14.4K
last updated: Aug 21, 2026

View on GitHub

Install

$ npx -y skills add orchestra-research/ai-research-skills --skill torchforge-rl-training --agent claude-code

Installs into .claude/skills of the current project.

Files

torchforge: PyTorch-Native Agentic RL Library

torchforge is Meta's PyTorch-native RL library that separates infrastructure concerns from algorithm concerns. It enables rapid RL research by letting you focus on algorithms while handling distributed training, inference, and weight sync automatically.

When to Use torchforge

Choose torchforge when you need:

Clean separation between RL algorithms and infrastructure
PyTorch-native abstractions (no Ray dependency)
Easy algorithm experimentation (GRPO, DAPO, SAPO in ~100 lines)
Scalable training with Monarch actor system
Integration with TorchTitan for model parallelism

Consider alternatives when:

You need production-ready stability → use miles or verl
You want Megatron-native training → use slime
torchforge is experimental and APIs may change

Key Features

Algorithm isolation: Implement RL algorithms without touching infrastructure
Scalability: From single GPU to thousands via Monarch
Modern stack: TorchTitan (training), vLLM (inference), TorchStore (sync)
Loss functions: GRPO, DAPO, CISPO, GSPO, SAPO built-in