Stable Baselines3

This gives Claude the knowledge to work with Stable Baselines3, the PyTorch RL library. It covers training agents with PPO, SAC, DQN and other algorithms, building custom Gym environments with proper validation, setting up vectorized environments for parallel training, and implementing callbacks for monitoring. The skill includes actual code patterns for common workflows like saving models, evaluating policies, and recording videos. Use this when you need to prototype RL experiments, train agents on custom tasks, or integrate reinforcement learning into a project without digging through docs. The guidance on environment constraints and vectorized env gotchas is genuinely helpful since those trip people up constantly.

skills.sh installs: 338
GitHub stars: 33.2K
last updated: Aug 21, 2026

View on GitHub

Install

$ npx -y skills add davila7/claude-code-templates --skill stable-baselines3 --agent claude-code

Installs into .claude/skills of the current project.

Files

Stable Baselines3

Overview

Stable Baselines3 (SB3) is a PyTorch-based library providing reliable implementations of reinforcement learning algorithms. This skill provides comprehensive guidance for training RL agents, creating custom environments, implementing callbacks, and optimizing training workflows using SB3's unified API.

Core Capabilities

1. Training RL Agents

Basic Training Pattern:

import gymnasium as gym
from stable_baselines3 import PPO

# Create environment
env = gym.make("CartPole-v1")

# Initialize agent
model = PPO("MlpPolicy", env, verbose=1)

# Train the agent
model.learn(total_timesteps=10000)

# Save the model
model.save("ppo_cartpole")

# Load the model (without prior instantiation)
model = PPO.load("ppo_cartpole", env=env)

Important Notes:

total_timesteps is a lower bound; actual training may exceed this due to batch collection
Use model.load() as a static method, not on an existing instance
The replay buffer is NOT saved with the model to save space

SKILL.md on GitHub