Plugin Audit

Plugin Audit

Audit how well vercel-plugin skill injection performs on real-world Claude Code sessions.

Workflow

1. Locate conversation logs

Find JSONL conversation logs for a target project:

ls -lt ~/.claude/projects/-Users-*-<project-name>/*.jsonl

The path uses the project's absolute path with slashes replaced by hyphens and a leading hyphen.

2. Extract tool calls

Parse the JSONL log to extract all tool_use entries. Each line is a JSON object with message.content[] containing type: "tool_use" blocks. Extract name and input fields. Group by tool type (Bash, Read, Write, Edit).

3. Test hook matching

Use the exported pipeline functions directly — do NOT shell out to the hook script for each test. Import from the hooks directory:

import { loadSkills, matchSkills } from "./hooks/pretooluse-skill-inject.mjs";
import { createLogger } from "./hooks/logger.mjs";

Call loadSkills() once, then matchSkills(toolName, toolInput, compiledSkills) for each tool call. This is fast and gives exact match results.

4. Identify gaps