Pufferlib

If you're doing reinforcement learning and hitting performance walls, this is worth checking out. PufferLib gets you 2-10x speedups over standard implementations through optimized vectorization and parallel environment simulation, hitting millions of steps per second. It's built around an efficient PPO implementation and works with all the usual frameworks (Gymnasium, PettingZoo, Atari, Procgen). The multi-agent support is native rather than bolted on, which matters if you're working with those environments. Fair warning: the learning curve is steeper than stable-baselines3, so this makes sense when you actually need the speed or are scaling up experiments. The documentation covers training loops, custom environments, and distributed setups across the reference files.

skills.sh installs: 2K
GitHub stars: 56.6K
last updated: Aug 21, 2026

View on GitHub

Install

$ npx -y skills add k-dense-ai/scientific-agent-skills --skill pufferlib --agent claude-code

Installs into .claude/skills of the current project.

Files

PufferLib

Choose the version before choosing an API. Reviewed 2026-10-01:

Profile	Status	Main use
Native source `5.0`	Current default branch and live documentation	C/CUDA environments, native trainer; CPU evaluation only
`pufferlib==3.0.0`	Latest PyPI release, published 2025-06-23; sdist only	Python/Gymnasium/PettingZoo adaptation and Torch PuffeRL
Pinned source `4.0`	Historical snapshot	C Ocean interface with an optional Torch fallback

For 5.0, read references/native-5.md. The reviewed revision is 6ffa5b10dbbbe4d1e8288367c7d9d3acd3bad4a2. Its CLI is ./puffer train after building an environment, not puffer train ENV_NAME. There is no 5.0 Python emulation/vector API or --slowly fallback.

SKILL.md on GitHub