Stable Baselines3

This is your go-to for production RL if you want battle-tested algorithms without reinventing the wheel. You get PPO, SAC, DQN, TD3, DDPG, and A2C with a scikit-learn style API that actually works. The skill covers the full workflow: training agents, building custom Gymnasium environments, vectorizing for parallel rollouts, and using callbacks for checkpointing and early stopping. It's built on PyTorch 2.3+ and designed for single-agent scenarios. The documentation is clear about constraints, like how SB3 doesn't support Discrete spaces with non-zero starts and normalizes uint8 images automatically. If you need massive parallel training or multi-agent systems, you'll want pufferlib instead, but for standard RL experiments and quick prototyping, this is the reliable choice.

skills.sh installs: 2.1K
GitHub stars: 56.6K
last updated: Aug 21, 2026

View on GitHub

Install

$ npx -y skills add k-dense-ai/scientific-agent-skills --skill stable-baselines3 --agent claude-code

Installs into .claude/skills of the current project.

Files

Stable Baselines3

Overview

Stable Baselines3 (SB3) is a PyTorch-based library providing reliable implementations of reinforcement learning algorithms. This skill provides comprehensive guidance for training RL agents, creating custom environments, implementing callbacks, and optimizing training workflows using SB3's unified API.

Current upstream: SB3 2.9.0 (June 15, 2026). Docs: stable-baselines3.readthedocs.io.

Installation

Tested against stable-baselines3 2.9.0. Requires Python 3.10+ (3.9 dropped in 2.8.0) and PyTorch >= 2.8.

# Basic installation
uv pip install "stable-baselines3==2.9.0"

# With extra dependencies (TensorBoard, ale-py for Atari, etc.)
uv pip install "stable-baselines3[extra]==2.9.0"

The 2.9.0 release supports Gymnasium >=0.29.1,<2.0; this review exercised Gymnasium 1.3.0, PyTorch 2.14.1 and Python 3.13 on CPU. pandas/matplotlib are now optional extras. Installation requires network unless packages are cached; local RL training needs no credentials or service endpoints.

On zsh, quote brackets: uv pip install 'stable-baselines3[extra]==2.9.0'.

SKILL.md on GitHub