Vercel Plugin Eval

This is a QA harness for testing the Vercel plugin end to end. It launches real Claude Code sessions in WezTerm, tails debug logs, and verifies all eight hook types fire correctly with proper deduplication. The skill walks you through creating timestamped test projects, monitoring skill injection counts against atomic claim files in tmpdir, and checking that PostToolUse validation catches real problems like API key bypasses. It includes a scenario design guide targeting different coverage zones, from AI chat apps to monorepos, with call outs for the eight hard to trigger skills that need explicit mentions. You write results to a coverage matrix showing which hooks and skills fired across sessions. Useful if you're maintaining the plugin or debugging why certain skills don't inject when expected.

skills.sh installs: 206
GitHub stars: 183
last updated: Jun 3, 2026

View on GitHub

Install

$ npx -y skills add vercel-labs/vercel-plugin --skill vercel-plugin-eval --agent claude-code

Installs into .claude/skills of the current project.

Files

Plugin Eval

Launch real Claude Code sessions with the plugin installed, monitor debug logs in real-time, and verify every hook fires correctly with proper dedup.

DO NOT (Hard Rules)

DO NOT use claude --print or -p — hooks don't fire, no files created
DO NOT use --dangerously-skip-permissions
DO NOT create projects in /tmp/ — always use ~/dev/vercel-plugin-testing/
DO NOT manually wire hooks or create settings.local.json — use npx add-plugin
DO NOT set CLAUDE_PLUGIN_ROOT manually
DO NOT use bash -c in WezTerm — use /bin/zsh -ic
DO NOT use full path to claude — use the x alias
DO NOT write eval scripts — do everything as Bash tool calls in the conversation

Copy the exact commands below. Do not improvise.

Quick Start

Always append a timestamp to directory names so reruns don't overwrite old projects:

# 1. Create test dir & install plugin (with timestamp)
TS=$(date +%Y%m%d-%H%M)
SLUG="my-eval-$TS"
mkdir -p ~/dev/vercel-plugin-testing/$SLUG
cd ~/dev/vercel-plugin-testing/$SLUG
npx add-plugin https://github.com/vercel/vercel-plugin -s project -y

SKILL.md on GitHub