Benchmark Sandbox

This runs your vercel-plugin benchmark scenarios inside ephemeral Firecracker microVMs instead of local terminals. Each sandbox gets a fresh Claude Code install, runs a three-phase pipeline (build, verify with agent-browser, deploy to Vercel), and scores each phase with structured JSON via a separate Haiku pass. The architecture is battle-tested: snapshots preserve both files and npm globals, port 3000 maps to a public URL at creation time, and extendTimeout keeps sandboxes alive overnight if you need it. You get concurrency control, per-phase timeouts, and the ability to load scenarios from JSON instead of hardcoding them. The real win is isolating each eval run so one broken scenario doesn't poison your local environment, plus you can parallelize up to 10 at once.

skills.sh installs: 181
GitHub stars: 183
last updated: Jun 3, 2026

View on GitHub

Install

$ npx -y skills add vercel-labs/vercel-plugin --skill benchmark-sandbox --agent claude-code

Installs into .claude/skills of the current project.

Files

Benchmark Sandbox — Remote Eval via Vercel Sandboxes

Run benchmark scenarios inside Vercel Sandboxes — ephemeral Firecracker microVMs with node24. Each sandbox gets a fresh Claude Code + Vercel CLI + agent-browser install, the local vercel-plugin uploaded, and runs a 3-phase eval pipeline:

Phase 1 (BUILD): Claude Code builds the app with --dangerously-skip-permissions --debug
Phase 2 (VERIFY): A follow-up Claude Code session uses agent-browser to walk through user stories, fixing issues until all pass (20 min timeout)
Phase 3 (DEPLOY): A third Claude Code session links to vercel-labs, runs vercel deploy, and fixes build errors (up to 3 retries). Deployed apps have deployment protection enabled by default.

Skills are tracked across all 3 phases — each phase may trigger additional skill injections as new files/patterns are created. After each phase, a haiku structured scoring step (claude -p --json-schema --model haiku) evaluates the results as structured JSON.