React app development - app-v2, standalone tools, Vercel deployment, and PolicyEngine-specific patterns

version
4.1.1
GitHub stars
31

Install

/plugin install app-development@policyengine-claude

Run it inside Claude Code after adding the marketplace with /plugin marketplace add PolicyEngine/policyengine-claude.

Contents

Commands 6

Create new component

Builds one or more new components for the @policyengine/ui-kit library. Handles the full lifecycle: research existing patterns, build with design tokens, validate token usage, write tests, preview visually, and open a PR.

Prerequisites

The policyengine-ui-kit repo must be cloned locally. If not found, clone it:

gh repo clone PolicyEngine/policyengine-ui-kit

All work happens in the policyengine-ui-kit repo directory.

Step 1: Gather requirements

Use AskUserQuestion to ask the user:

  1. Component description — What component(s) do you need? Describe their purpose, behavior, and any variants.

Parse the description to identify:

  • Component names (PascalCase, e.g., Tooltip, ProgressBar, Accordion)
  • Which category they belong to: primitives, layout, inputs, display, or charts
  • Expected props, variants, and states

Step 2: Research app-v2 for similar components

Before building from scratch, search PolicyEngine/policyengine-app-v2 for similar implementations:

# Search app-v2 for components with similar names or purposes
find /path/to/policyengine-app-v2/app/src -name "*.tsx" | xargs grep -li "COMPONENT_NAME_PATTERN"

# Check how app-v2 composes the same component type (radix-ui + Tailwind + CVA)
grep -r "COMPONENT_TYPE" /path/to/policyengine-app-v2/app/src --include="*.tsx" -l
grep -r "COMPONENT_TYPE" /path/to/policyengine-app-v2/website/src --include="*.tsx" -l

If similar components exist in app-v2:

  • Study their props interface, variant patterns, and behavior
  • Adapt the design and UX, rebuilt on the ui-kit's Tailwind v4 + CVA stack

If nothing similar exists in app-v2:

  • Research common patterns for that component type
  • Design props and variants that are consistent with existing ui-kit components

Step 3: Build the component(s)

Tech stack (mandatory):

  • React 19 with forwardRef
  • TypeScript with strict types
  • Tailwind CSS v4 with standard class names (no prefix)
  • CVA (class-variance-authority) for variant management
  • cn() utility from @/utils/cn for class merging
  • Design tokens from ui-kit's theme.css (colors, spacing, typography, radius)

File structure for each component:

src/<category>/ComponentName.tsx

Component template:

import { cva, type VariantProps } from 'class-variance-authority';
import { forwardRef, type HTMLAttributes } from 'react';
import { cn } from '../utils/cn';

const componentVariants = cva(
  'base-classes-here',
  {
    variants: {
      variant: {
        default: 'variant-classes',
        // ... more variants
      },
      size: {
        sm: 'size-classes',
        md: 'size-classes',
        lg: 'size-classes',
      },
    },
    defaultVariants: { variant: 'default', size: 'md' },
  },
);

export interface ComponentNameProps
  extends HTMLAttributes<HTMLDivElement>,
    VariantProps<typeof componentVariants> {
  // Additional props here
}

export const ComponentName = forwardRef<HTMLDivElement, ComponentNameProps>(
  ({ variant, size, className, children, ...props }, ref) => (
    <div
      ref={ref}
      className={cn(componentVariants({ variant, size }), className)}
      {...props}
    >
      {children}
    </div>
  ),
);
ComponentName.displayName = 'ComponentName';

Rules:

  • ALL colors, spacing, font sizes, and radii must reference design tokens — no hardcoded hex values, no arbitrary pixel values
  • Use semantic classes (bg-primary, text-foreground) or brand palette classes (bg-teal-500, text-gray-600)
  • Export the component and its props interface from the category barrel (src/<category>/index.ts)
  • Export from the main barrel (src/index.ts)
  • Build ALL components the user requested — do not skip any

Step 4: Design token validation agent

Launch the Dashboard Design Validator Agent (agents/dashboard/dashboard-design-validator.md) to audit every component file created in Step 3.

The agent will:

  1. Scan all new component files for hardcoded values
  2. Replace hardcoded colors, spacing, font sizes, and radii with token-based Tailwind classes
  3. Report what was replaced and what was kept

Review the agent's output. If it made changes, verify they look correct.

Step 5: Add components to a preview page

Create or update demo/Demo.tsx to include a new section showcasing ALL new components with realistic example data.

// Add a new section for each component
<section>
  <h2 className="text-2xl font-bold mb-4">ComponentName</h2>
  <div className="flex flex-wrap gap-3">
    {/* Render every variant, size, and state */}
    <ComponentName variant="default">Example</ComponentName>
    <ComponentName variant="primary" size="lg">Large primary</ComponentName>
    {/* ... all combinations */}
  </div>
</section>

Important: Include ALL new components on the preview page, not just some.

Step 6: Launch preview and iterate with user

Start the demo dev server:

cd /path/to/policyengine-ui-kit
bun run dev:demo

Tell the user:

The demo server is running at http://localhost:5173/demo/index.html — scroll to the new component section to preview your components.

Then use AskUserQuestion to ask:

Do the components look correct? Please review and either approve or describe changes you'd like.

This is iterative. If the user requests changes:

  1. Apply the requested modifications
  2. The dev server will hot-reload automatically
  3. Ask the user to review again
  4. Repeat until the user approves

Do NOT proceed to Step 7 until the user explicitly approves the components.

Step 7: Component test writer agent

After user approval, launch the Component Test Writer Agent (agents/app/component-test-writer.md) to write unit tests for ALL new components.

The agent will:

  1. Read each new component file
  2. Write comprehensive test files (rendering, props, variants, interaction, ref forwarding)
  3. Place tests next to components (src/<category>/ComponentName.test.tsx)
  4. Run bun run test and fix any failures

Verify all tests pass:

cd /path/to/policyengine-ui-kit
bun run test

If tests fail, fix them before proceeding.

Step 8: Create changelog fragment

Add a towncrier changelog fragment:

echo "Add <ComponentName>, <ComponentName2>, ... components." > changelog.d/add-COMPONENT_NAME.added.md

Step 9: Commit, push, and open PR

# Create feature branch
git checkout -b add-COMPONENT_NAME-component

# Stage all new and modified files
git add src/<category>/ComponentName.tsx
git add src/<category>/ComponentName.test.tsx
git add src/<category>/index.ts
git add src/index.ts
git add demo/Demo.tsx
git add changelog.d/add-COMPONENT_NAME.added.md

# Commit
git commit -m "Add ComponentName component with tests"

# Push
git push -u origin add-COMPONENT_NAME-component

Create a PR with details about the new components and their test coverage:

gh pr create --repo PolicyEngine/policyengine-ui-kit \
  --title "Add ComponentName component" \
  --body "$(cat <<'EOF'
## Summary
- Add `ComponentName` component to `@policyengine/ui-kit`
- <Describe each component, its variants, and purpose>

## Components added
- `ComponentName` — <brief description>
  - Variants: <list variants>
  - Props: <list key props>

## Testing
- Unit tests written with Vitest + React Testing Library
- Tests cover: rendering, all variants, props, interaction, ref forwarding
- All tests passing

## Design token compliance
- All colors reference design tokens (no hardcoded hex)
- All spacing uses standard Tailwind classes
- All typography uses token scale
- Validated by dashboard-design-validator agent

## Test plan
- [ ] Run `bun run dev:demo` and verify components render correctly
- [ ] Run `bun run test` and verify all tests pass
- [ ] Review component API matches existing ui-kit conventions

🤖 Generated with [Claude Code](https://claude.com/claude-code)
EOF
)"

Report the PR URL to the user.

Reference

See these skills and agents for detailed guidance:

  • policyengine-design — Design token values and usage
  • policyengine-app — app-v2 component patterns to reference
  • agents/dashboard/dashboard-design-validator.md — Automated design token compliance
  • agents/app/component-test-writer.md — Automated test writing

Creating PR with CI Verification

IMPORTANT: This command solves the common problem where Claude creates a PR, says "I'll wait for CI", then gives up. This command ACTUALLY waits for CI completion.

Workflow

This command will:

  1. ✅ Create PR as draft
  2. ✅ Wait for CI checks to complete (actually wait, not give up)
  3. ✅ Mark as ready for review if CI passes
  4. ✅ Automatically trigger /fix-pr if CI fails (new!)

Step 1: Determine Current Branch and Changes

# Get current branch
CURRENT_BRANCH=$(git branch --show-current)

# Get base branch (usually main or master)
BASE_BRANCH=$(git remote show origin | grep 'HEAD branch' | cut -d' ' -f5)

# Verify we're not on main/master
if [[ "$CURRENT_BRANCH" == "main" || "$CURRENT_BRANCH" == "master" ]]; then
  echo "ERROR: Cannot create PR from main/master branch"
  echo "Create a feature branch first: git checkout -b feature-name"
  exit 1
fi

# Check if branch is pushed
git rev-parse --verify origin/$CURRENT_BRANCH > /dev/null 2>&1
if [ $? -ne 0 ]; then
  echo "Branch not pushed yet. Pushing now..."
  git push -u origin $CURRENT_BRANCH
fi

Step 2: Gather PR Context

Run in parallel to gather information:

# 1. Git status
git status

# 2. Diff since divergence from base
git diff ${BASE_BRANCH}...HEAD

# 3. Commit history for this branch
git log ${BASE_BRANCH}..HEAD --oneline

# 4. Check if PR already exists
gh pr view --json number,state,isDraft 2>/dev/null

If PR already exists:

EXISTING_PR=$(gh pr view --json number --jq '.number')
echo "PR #$EXISTING_PR already exists for this branch"
echo "Use /review-program $EXISTING_PR or /fix-pr $EXISTING_PR instead"
exit 0

Step 3: Create PR Title and Body

Based on the changes (read from git diff and commit history):

Title: Concise summary of changes (50 chars max)

Body format:

## Summary
- Bullet point 1
- Bullet point 2

## Changes
- Specific change 1
- Specific change 2

## Testing
- Test approach
- Verification steps

## Checklist
- [ ] Tests pass locally
- [ ] Code formatted (make format)
- [ ] Changelog entry created (if applicable)

---

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

Step 4: Create PR as Draft

CRITICAL: Always create as draft initially.

# Create PR as draft
# CRITICAL: Use --repo flag to create PR in upstream repo from fork
# This creates cross-fork PR: your-fork:branch → PolicyEngine:master
gh pr create --repo PolicyEngine/policyengine-us \
  --title "PR Title Here" \
  --body "$(cat <<'EOF'
PR body here
EOF
)" \
  --draft

# Get PR number
PR_NUMBER=$(gh pr view --json number --jq '.number')
echo "Created draft PR #$PR_NUMBER"
echo "URL: $(gh pr view --json url --jq '.url')"

Step 5: Wait for CI Checks (DO NOT GIVE UP!)

This is where Claude usually fails. DO NOT say "I'll check back later".

Instead, ACTUALLY wait using a polling loop with NO timeout:

echo "Waiting for CI checks to complete..."
echo "Typical CI times by repository:"
echo "  - policyengine-app: 3-6 minutes"
echo "  - policyengine-uk: 9-12 minutes"
echo "  - policyengine-api: 30 minutes"
echo "  - policyengine-us: 60-75 minutes (longest)"
echo ""
echo "I will poll every 15 seconds until all checks complete. No time limit."

POLL_INTERVAL=15  # Check every 15 seconds
ELAPSED=0

while true; do
  # Get check status
  CHECKS_JSON=$(gh pr checks $PR_NUMBER --json name,status,conclusion)

  # Count checks
  TOTAL=$(echo "$CHECKS_JSON" | jq '. | length')

  if [ "$TOTAL" -eq 0 ]; then
    echo "No CI checks found yet. Waiting for checks to start..."
    sleep $POLL_INTERVAL
    ELAPSED=$((ELAPSED + POLL_INTERVAL))
    continue
  fi

  # Count by status
  COMPLETED=$(echo "$CHECKS_JSON" | jq '[.[] | select(.status == "COMPLETED")] | length')
  FAILED=$(echo "$CHECKS_JSON" | jq '[.[] | select(.conclusion == "FAILURE")] | length')

  echo "[$ELAPSED s] CI Checks: $COMPLETED/$TOTAL completed, $FAILED failed"

  # If all completed
  if [ "$COMPLETED" -eq "$TOTAL" ]; then
    if [ "$FAILED" -eq 0 ]; then
      echo "✅ All CI checks passed after $ELAPSED seconds!"
      break
    else
      echo "❌ Some CI checks failed after $ELAPSED seconds."
      # Show failures
      gh pr checks $PR_NUMBER
      break
    fi
  fi

  # Continue waiting (no timeout!)
  sleep $POLL_INTERVAL
  ELAPSED=$((ELAPSED + POLL_INTERVAL))
done

Important: No timeout means we actually wait as long as needed. Population simulations can take 30+ minutes.

Step 6: Mark Ready for Review (If CI Passed) or Auto-Fix (If CI Failed)

if [ "$FAILED" -eq 0 ] && [ "$COMPLETED" -eq "$TOTAL" ]; then
  echo "Marking PR as ready for review..."
  gh pr ready $PR_NUMBER

  echo ""
  echo "✅ PR #$PR_NUMBER is ready for review!"
  echo "URL: $(gh pr view $PR_NUMBER --json url --jq '.url')"
  echo ""
  echo "All CI checks passed:"
  gh pr checks $PR_NUMBER
else
  echo ""
  echo "❌ CI checks failed. PR remains as draft."
  echo "URL: $(gh pr view $PR_NUMBER --json url --jq '.url')"
  echo ""
  echo "Failed checks:"
  gh pr checks $PR_NUMBER
  echo ""
  echo "🔧 Automatically triggering /fix-pr to address CI failures..."
  echo ""
fi

If CI failed, automatically trigger fix-pr command:

After detecting CI failures, invoke the /fix-pr command to automatically attempt to fix the issues. This ensures the PR doesn't stay broken and saves user intervention.

The /fix-pr command will:

  1. Analyze the CI failures
  2. Apply appropriate fixes (formatting, linting, test failures)
  3. Push the fixes
  4. Wait for CI to pass again
  5. Mark PR as ready once all checks pass

Key Differences from Default Behavior

Old way (Claude gives up):

Claude: "I've created the PR. CI will take a while, I'll check back later..."
[Chat ends, Claude never checks back]
User: Has to manually check CI and mark ready

New way (this command waits and auto-fixes):

Claude: "I've created the PR as draft. Now waiting for CI..."
[Actually polls CI status every 15 seconds]

Scenario A - CI passes:
Claude: "CI passed! Marking as ready for review."
[PR is ready, user doesn't have to do anything]

Scenario B - CI fails:
Claude: "CI failed. Automatically triggering /fix-pr..."
[Analyzes failures, applies fixes, pushes changes]
[Waits for CI again, marks ready when passes]
[User doesn't have to do anything]

Error Handling

If CI fails:

The command automatically triggers /fix-pr to handle failures. No manual intervention needed in most cases.

The auto-fix will handle:

  • Linting errors: make format && git push
  • Import sorting: isort
  • Test failures: Analyzed and delegated to appropriate agents
  • Changelog validation: Creates missing entries
  • Reference validation: Fixes parameter references

If no CI configured:

if [ "$TOTAL" -eq 0 ] && [ $ELAPSED -gt 60 ]; then
  echo "⚠️ No CI checks detected after 60 seconds."
  echo "Repository may not have CI configured."
  echo "Marking PR as ready for manual review."
  gh pr ready $PR_NUMBER
fi

Usage Examples

Basic:

# Make changes, commit
git add .
git commit -m "Add feature"

# Create PR (command waits for CI)
/create-pr

With custom title:

# Command can accept optional arguments
/create-pr "Add California EITC implementation"

Checking existing PR:

# If PR exists, command tells you and suggests alternatives
/create-pr
# Output: "PR #123 exists. Use /review-program 123 or /fix-pr 123"

Integration with Other Commands

After /encode-policy-v2 or /fix-pr:

# These commands might push code
# Use /create-pr to create PR and wait for CI
/create-pr

After fixing CI issues:

# Fix issues locally
make format
git add .
git commit -m "Fix linting"
git push

# Command will detect existing PR and re-check CI
/create-pr  # "PR #123 exists, checking CI status..."

CI Duration Expectations

By repository (based on actual data):

  • policyengine-app: 3-6 minutes (fast)
  • policyengine-uk: 9-12 minutes (medium)
  • policyengine-api: ~30 minutes (slow)
  • policyengine-us: 60-75 minutes (very slow - population simulations)

The command has no timeout - it will wait as long as needed, even for policyengine-us's 75-minute CI runs.

Why This Works

Problem: Claude's internal timeout or context leads it to give up

Solution:

  1. ✅ Explicit polling loop (Claude sees the code will wait)
  2. ✅ Clear status updates (Claude knows it's still working)
  3. ✅ Timeout is explicit (Claude knows how long to wait)
  4. ✅ Fallback handling (What to do if timeout occurs)

Key insight: By having the command contain the waiting logic in bash, Claude executes it and actually waits instead of just saying "I'll wait."

Notes

This command is essential for:

  • Automated PR workflows
  • CI-dependent merges
  • Team workflows requiring CI validation
  • Reducing manual PR management

Use this instead of:

  • Manual gh pr create followed by manual CI checking
  • Asking Claude to "wait for CI" (which it can't do reliably)
  • Creating ready PRs before CI passes

New interactive tool scaffold

Creates a complete project for a standalone PolicyEngine interactive tool that embeds in policyengine.org.

When to use vs /create-dashboard

  • /new-tool (this command) — one-purpose calculator or widget. Ships with design tokens + embedding boilerplate; you write the specific tool logic. Examples: salternative, aca-calc, marriage-penalty-calculator.
  • /create-dashboard — multi-page policy analysis app with multi-agent plan/scaffold/implement/validate. Use when there are multiple views, comparison modes, filters, or the dashboard is answering a research question rather than performing a calculation.

Step 1: Gather requirements

Ask the user for:

  1. Tool name (kebab-case, e.g., marriage, aca-calc, salary-sacrifice-tool)
  2. Countries (us, uk, or both)
  3. Data pattern — how the tool gets model results:
    • A) Precomputed — Static JSON or CSV shipped with the app (best for finite parameter spaces)
    • B) Dashboard API — The page calls www.policyengine.org's dashboard API through @policyengine/api-client (best for household calculators). Never call api.policyengine.org from the page: the Public API is becoming callable only by PolicyEngine's servers (policyengine-api-access skill)
    • C) Custom Modal API — Python serverless function with the top-level policyengine[us] / policyengine[uk] package (only when the dashboard API doesn't cover it and precomputing can't: microsimulation output beyond its economy comparison, a dataset the Public API doesn't serve, or a newer model release while the policyengine package's model bundle isn't updated weekly; a structural reform goes into the country package as a contributed reform instead)
  4. Brief description of what the tool calculates

Step 2: Create the project

# Create Next.js 14 + Tailwind project
bunx create-next-app@14 TOOL_NAME --js --app --tailwind --eslint --no-src-dir --import-alias "@/*"
cd TOOL_NAME

# Install dependencies
bun add @policyengine/ui-kit recharts
bun add -D vitest

Copy the favicon:

mkdir -p public
cp node_modules/@policyengine/ui-kit/src/assets/logos/policyengine/teal-square.svg public/favicon.svg

If using code highlighting:

bun add prism-react-renderer

Step 3: Generate project files

Create the following files with the content specified below. Replace TOOL_NAME, TOOL_TITLE, DESCRIPTION, and COUNTRY_ID with actual values.

app/layout.jsx

import "./globals.css";

export const metadata = {
  title: "TOOL_TITLE | PolicyEngine",
  description: "DESCRIPTION",
  icons: { icon: "/favicon.svg" },
};

export default function RootLayout({ children }) {
  return (
    <html lang="en">
      <head>
        <link
          href="https://fonts.googleapis.com/css2?family=Inter:wght@400;500;600;700&display=swap"
          rel="stylesheet"
        />
      </head>
      <body>{children}</body>
    </html>
  );
}

app/globals.css

@import "tailwindcss";
@import "@policyengine/ui-kit/theme.css";

body {
  font-family: var(--font-sans);
  color: var(--foreground);
  background: var(--background);
  -webkit-font-smoothing: antialiased;
  -moz-osx-font-smoothing: grayscale;
}

The single @import "@policyengine/ui-kit/theme.css" provides all design tokens (colors, spacing, typography, chart colors) as CSS variables that Tailwind 4 picks up automatically. No manual @theme block needed.

app/page.jsx

"use client";

import { useState } from "react";
import { DashboardShell, Header, logos } from "@policyengine/ui-kit";

function getCountryFromHash() {
  if (typeof window === "undefined") return "us";
  const params = new URLSearchParams(window.location.hash.slice(1));
  return params.get("country") || "us";
}

export default function Home() {
  const [countryId] = useState(getCountryFromHash());
  const isEmbedded =
    typeof window !== "undefined" && window.self !== window.top;

  function updateHash(params) {
    const p = new URLSearchParams();
    // Add your params here, e.g.: p.set("income", params.income);
    if (countryId !== "us" && !isEmbedded) p.set("country", countryId);
    const hash = `#${p.toString()}`;
    window.history.replaceState(null, "", hash);
    if (isEmbedded) {
      window.parent.postMessage({ type: "hashchange", hash }, "*");
    }
  }

  function getShareUrl() {
    const hash = window.location.hash;
    if (isEmbedded) {
      return `https://policyengine.org/${countryId}/TOOL_NAME${hash}`;
    }
    return window.location.href;
  }

  return (
    <DashboardShell>
      <Header
        variant="dark"
        logo={<img src={logos.whiteWordmark} alt="PolicyEngine" className="h-5" />}
      >
        <span className="ml-2 font-bold text-white">TOOL_TITLE</span>
      </Header>
      <main className="max-w-6xl mx-auto p-6">
        {/* Your tool UI goes here */}
      </main>
    </DashboardShell>
  );
}

vercel.json

{
  "framework": "nextjs"
}

Step 4: Data pattern boilerplate

Prefer Pattern B (the dashboard API) unless the tool needs outputs the dashboard API can't give and precomputing can't cover. A structural reform goes into the country package as a contributed reform rather than into a Pattern C backend, rate limits are not reasons for Pattern C, and the Public API's model version is one only while the policyengine package's model bundle isn't updated weekly (policyengine-tools, "Data and backend patterns"). Pattern C is significantly more complex and should be the last resort.

Based on the user's choice, add the appropriate data fetching code.

For Pattern B (the dashboard API): run bun add @policyengine/api-client, then create lib/api.js:

import { createPolicyEngineApiClient } from "@policyengine/api-client";

// www.policyengine.org's dashboard API; its server adds PolicyEngine's credential.
const policyengine = createPolicyEngineApiClient();

export function calculate(countryId, household, signal) {
  // Resolves with the calculated household; throws PolicyEngineApiError on a refusal.
  return policyengine.calculateHousehold({ country: countryId, household }, { signal });
}

The dashboard API answers pages on the tool's www.policyengine.org path, on localhost, on the team's Vercel previews and on the tool's own *.vercel.app host once its rewrite is in appZoneRoutes.ts. The tool's <project>-policy-engine.vercel.app host is one of the team's Vercel hosts, so it is allowed from the first deploy, before the rewrite exists. A tool served from any other host, such as a Modal URL or a custom domain, needs an exact entry for that host in the allow list in website/src/lib/dashboardApi/cors.ts (PolicyEngine/policyengine-app-v2), deployed before its pages call the dashboard API; until then they get 403, which the browser shows as a CORS error. The policyengine-api-access skill has the other operations (policies, economy comparisons with polling, metadata) and the errors.

For Pattern C (Custom Modal API — gateway + polling):

Pattern C uses a three-file backend structure mirroring policyengine-api-v2's simulation service: a standalone image setup, a worker app, and pure simulation logic. A lightweight gateway manages job submission/polling. This avoids Modal's ~150s gateway timeout and a common crash-loop where module-level imports fail.

First, look up the latest version of the top-level policyengine package from PyPI (must be >=5.0.1). Do NOT guess or use a version from memory:

pip index versions policyengine 2>/dev/null | head -1

Install policyengine[us] (or policyengine[uk]) rather than a bare country package — the extra pins an exactly-matched country model and carries the certified data-bundle manifest.

Create backend/_image_setup.py (standalone snapshot function, no package imports at module level):

def snapshot_models():
    """Pre-load models at image build time for fast cold starts."""
    import logging
    logging.basicConfig(level=logging.INFO)
    logger = logging.getLogger(__name__)
    logger.info("Pre-loading tax-benefit system...")
    from policyengine_us import CountryTaxBenefitSystem  # or policyengine_uk
    CountryTaxBenefitSystem()
    # Tools with population/microsimulation endpoints: also run
    # `import policyengine as pe; pe.us.managed_microsimulation()` here so the
    # certified dataset is cached in the image (adds GBs — skip for household-only).
    logger.info("Models pre-loaded into image snapshot")

Create backend/simulation.py (pure logic, policyengine at module level — captured in snapshot):

from policyengine_us import Simulation  # household-level; snapshotted at build time

def run_compute(params: dict) -> dict:
    sim = Simulation(situation=params["household"])
    return {"result": float(sim.calculate("variable_name", 2025).sum())}

If the tool needs population-scale results (statewide/national impacts), compute them through policyengine.py's managed path — import policyengine as pe; sim = pe.us.managed_microsimulation(reform=...) (policyengine>=5.0.1) — never a directly-imported country-package Microsimulation, whose default dataset can lag the certified bundle. The managed constructor returns the same MicroSeries .calc() surface and records provenance on sim.policyengine_bundle; see the policyengine skill.

Create backend/app.py (worker app, only modal at module level):

import modal
from pathlib import Path
from _image_setup import snapshot_models

_BACKEND_DIR = Path(__file__).parent
app = modal.App("TOOL_NAME-workers", tags={"project": "TOOL_NAME"})  # tag every app of the tool, so its bill adds up in one query
image = (
    modal.Image.debian_slim(python_version="3.11")
    .pip_install("policyengine[us]==LATEST_VERSION", "pydantic")  # pins country model + certified data bundle
    .run_function(snapshot_models)
    .add_local_file(str(_BACKEND_DIR / "simulation.py"), remote_path="/root/simulation.py")
)

# Household-scale default: a US household calculation peaked at 1.4 GB. For a population
# run, size from a measured run and declare it (policyengine-cloud-costs skill, rule RS1).
@app.function(image=image, cpu=1.0, memory=2048, timeout=600, max_containers=10, min_containers=0)
def compute(params: dict) -> dict:
    from simulation import run_compute
    return run_compute(params)

Create backend/modal_app.py (lightweight gateway, no policyengine):

import modal
from fastapi import FastAPI, HTTPException
from fastapi.middleware.cors import CORSMiddleware
from pydantic import BaseModel

app = modal.App("TOOL_NAME", tags={"project": "TOOL_NAME"})
gateway_image = modal.Image.debian_slim(python_version="3.11").pip_install("fastapi", "pydantic")
WORKER_APP = "TOOL_NAME-workers"
FUNCTION_MAP = {"calculate": "compute"}

web_app = FastAPI()
web_app.add_middleware(CORSMiddleware, allow_origins=["*"], allow_methods=["*"], allow_headers=["*"])

@web_app.post("/submit/{endpoint}")
def submit(endpoint: str, params: dict):
    if endpoint not in FUNCTION_MAP:
        raise HTTPException(status_code=404, detail=f"Unknown endpoint: {endpoint}")
    fn = modal.Function.from_name(WORKER_APP, FUNCTION_MAP[endpoint])
    call = fn.spawn(params)
    return {"job_id": call.object_id}

@web_app.get("/status/{job_id}")
def status(job_id: str):
    from modal.functions import FunctionCall
    call = FunctionCall.from_id(job_id)
    try:
        result = call.get(timeout=0)
        return {"status": "ok", "result": result}
    except TimeoutError:
        return {"status": "computing"}
    except Exception as e:
        return {"status": "error", "message": str(e)}

@app.function(image=gateway_image)
@modal.asgi_app()
def fastapi_app():
    return web_app

And lib/api.js (polling client):

const API_URL =
  process.env.NEXT_PUBLIC_API_URL ||
  "https://policyengine--TOOL_NAME-fastapi-app.modal.run";

export async function submitJob(endpoint, params) {
  const res = await fetch(`${API_URL}/submit/${endpoint}`, {
    method: "POST",
    headers: { "Content-Type": "application/json" },
    body: JSON.stringify(params),
  });
  if (!res.ok) throw new Error(`Submit failed: ${res.status}`);
  const data = await res.json();
  return data.job_id;
}

export async function pollStatus(jobId) {
  const res = await fetch(`${API_URL}/status/${jobId}`);
  if (!res.ok) throw new Error(`Status check failed: ${res.status}`);
  return res.json(); // { status: "computing" | "ok" | "error", result?, message? }
}

Step 5: Initialize git and deploy

# Initialize repo
git init
git add -A
git commit -m "Initial scaffold for TOOL_TITLE"

# Create GitHub repo under PolicyEngine org
gh repo create PolicyEngine/TOOL_NAME --public --source=.
git push -u origin main

# Deploy to Vercel
vercel link --scope policy-engine
vercel --prod --yes

If using Pattern C (Modal gateway + worker), first add the "Running costs" section from the policyengine-cloud-costs skill's templates/running-costs.md to the README, then:

unset MODAL_TOKEN_ID MODAL_TOKEN_SECRET
# Deploy worker first (includes image snapshot — first build takes ~5 min)
modal deploy backend/app.py
# Deploy gateway (lightweight job submission/polling)
modal deploy backend/modal_app.py
vercel env add NEXT_PUBLIC_API_URL production
# Enter: https://policyengine--TOOL_NAME-fastapi-app.modal.run
vercel --prod --force --yes --scope policy-engine

Step 6: Add pull-request CI

Create .github/workflows/ci.yml so every new tool gets build validation before deploy. Replace CHECKER_COMMIT_SHA with the output of gh api repos/PolicyEngine/policyengine-skills/commits/main --jq .sha, so the cost checker's rules change only when the tool's maintainers move that commit:

name: CI

on:
  pull_request:
  push:
    branches: [main, master]

jobs:
  ci:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: oven-sh/setup-bun@v2
      - name: Run PolicyEngine migration guardrails
        run: |
          curl -fsSL https://raw.githubusercontent.com/PolicyEngine/policyengine-skills/main/scripts/audit_next_migration.py -o /tmp/audit_next_migration.py
          python3 /tmp/audit_next_migration.py --root .
      - name: Check deploy settings for cost
        env:
          # A policyengine-skills commit; move it forward on purpose (policyengine-cloud-costs skill, section 4)
          CHECKER_COMMIT: CHECKER_COMMIT_SHA
        run: |
          curl -fsSL "https://raw.githubusercontent.com/PolicyEngine/policyengine-skills/${CHECKER_COMMIT}/skills/policyengine-cloud-costs/scripts/check_deploy_config.py" -o /tmp/check_deploy_config.py
          python3 /tmp/check_deploy_config.py .
      - run: bun install --frozen-lockfile
      - name: Run lint
        run: |
          if jq -e '.scripts.lint' package.json >/dev/null; then
            bun run lint
          else
            echo "No lint script"
          fi
      - name: Run tests
        run: |
          if jq -e '.scripts.test' package.json >/dev/null; then
            bun run test
          else
            echo "No test script"
          fi
      - run: bun run build

Every embeddable tool should have migration guardrails and the cost check, plus at least install + build on pull requests. If the repo has lint or test scripts, wire them into CI immediately rather than leaving the repo deploy-only.

Step 7: Register in apps.json

Add the tool entry to website/src/data/apps.json in policyengine-app-v2. Use the auto-assigned Vercel production URL (not a custom alias).

If the tool needs a rewrite/proxy path rather than a direct iframe source URL, update website/next.config.ts in the same repo as part of the launch.

Step 8: Verify

# Check deployment
curl -s -o /dev/null -w "%{http_code}" https://VERCEL_URL/

# Start dev server for local development
bun run dev

# Validate the production build locally
bun run build

# Run migration guardrails locally
curl -fsSL https://raw.githubusercontent.com/PolicyEngine/policyengine-skills/main/scripts/audit_next_migration.py | python3 - --root .

# Check deploy settings for cost
curl -fsSL https://raw.githubusercontent.com/PolicyEngine/policyengine-skills/main/skills/policyengine-cloud-costs/scripts/check_deploy_config.py | python3 - .

# Run lint/tests when the repo defines them
if jq -e '.scripts.lint' package.json >/dev/null; then bun run lint; fi
if jq -e '.scripts.test' package.json >/dev/null; then bun run test; fi

Reference

See these skills for detailed guidance:

  • policyengine-tools — Embedding, hash sync, country detection, Vercel deployment patterns
  • policyengine-api-access — The dashboard API for Pattern B, and server credentials for a backend that calls the Public API
  • policyengine-design — Token reference
  • policyengine-cloud-costs — Backend defaults, declared exceptions, the README's running costs, and the cost checks

SEO audit: $ARGUMENTS

Read-only. Run every check yourself in one pass — no subagents. Report findings; make no code changes.

Options

  • --url <URL> — audit a deployed page directly (fetch its served HTML).
  • --repo OWNER/NAME — shallow-clone to /tmp/seo-audit-target and audit its built index.html.
  • --local — print findings only; skip the GitHub-issue prompt.
  • No args — audit the current working directory's built output.

Step 1: Get the HTML the crawler sees

  • Deployed URL (--url, or a CNAME / vercel.json / Pages URL you locate): WebFetch it, or curl -sL <URL>. curl returns the raw pre-JS shell — that is what Googlebot indexes first.
  • Local: build if needed (bun install && bun run build, else npm), then read dist/index.html or build/index.html (check a frontend/ subdir too for monorepos).

Step 2: Meta, Open Graph, canonical (check directly)

  • <title> (unique, ≤60 chars) and <meta name="description"> (≤160 chars).
  • Open Graph: og:title, og:description, og:image (absolute URL), og:url, og:type.
  • Twitter: twitter:card, twitter:title, twitter:image.
  • <link rel="canonical"> present and absolute (see Step 4 for which URL).
  • Structured data (application/ld+json) where the page warrants it.

Step 3: Crawlability

  • curl -sL <origin>/robots.txt and /sitemap.xml — present, reachable, and the sitemap lists real URLs.
  • SPA shells (Vite/CRA) ship a near-empty <body>; confirm prerendering/SSG (or Next.js SSR) so crawlers receive real content and headings, not just a loading div.
  • Exactly one <h1>, sane heading hierarchy, and lang set on <html>.

Step 4: PolicyEngine-specific atoms

  1. Dual-mode canonical. PE tools run both standalone (Vercel/Pages) and embedded as an iframe/zone on policyengine.org. The canonical must point to the ONE indexable home (the standalone deploy) so the two copies don't compete as duplicate content.
  2. Multi-zone paths. When the tool is a path-mounted zone under policyengine.org, its canonical / og:url / sitemap URLs must use the public policyengine.org/<path> URL — never the raw Vercel host or _zones/ asset URL. See the policyengine-tools skill (multi-zone integration).
  3. .github.io Search Console (legacy note). Project sites under policyengine.github.io/<repo> can't be independently verified in Search Console (the org owns the apex), so their standalone indexing was always weak. New tools deploy to Vercel — treat any github.io target as legacy.

Step 5: Report

Group findings as Critical / Important / Suggestions, each paired with its fix. State the standalone URL, the embedding URL (if any), and the canonical strategy (present / missing / misconfigured).

Unless --local, offer to open a GitHub issue on the audited repo via AskUserQuestion. If yes:

gh issue create --repo "$REPO_FULL" \
  --title "SEO audit: {N} critical issues" \
  --body "$REPORT_BODY" --label "seo,audit"   # retry without --label if the labels don't exist

Report the issue URL.

Multi-zone Audit: $ARGUMENTS

READ-ONLY MODE: This command audits an existing PolicyEngine Next.js tool for multi-zone compliance and reports findings. It does NOT make code changes.

Use this to verify that a zone will work correctly behind policyengine.org, in its own Vercel preview, and in next dev.

Migration-era command. This exists to retrofit existing PolicyEngine Next.js tools (built before the multi-zone architecture was standardized) and to verify tools built outside the /create-dashboard flow. Once all PolicyEngine Next.js apps conform to the multi-zone architecture, this command can be removed — new apps created via /create-dashboard are already validated by the dashboard validators at scaffold time.

Scope

This command audits Next.js zone apps — tools that deploy as a zone behind policyengine-app-v2/website/. It does not apply to:

  • policyengine-app-v2 itself (the host — different rules)
  • Legacy iframe-embedded tools (not yet migrated to multi-zone)
  • Non-Next.js tools (Python/Modal dashboards, static GitHub Pages sites)

The underlying agent (multizone-validator) detects these cases and stops with a clear message.

Options

  • --repo OWNER/NAME — Audit a specific GitHub repo (clones it to /tmp)
  • --path ABSOLUTE_PATH — Audit a local path (e.g., a worktree or sibling clone)
  • No arguments — Audit the current working directory

Examples

/audit-multizone                                        # Audit current working directory
/audit-multizone --repo PolicyEngine/keep-your-pay-act  # Clone and audit a GitHub repo
/audit-multizone --path /Users/me/Work/household-api-docs

Phase 0: Parse arguments

Parse $ARGUMENTS:
- REPO_ARG: value after --repo if present
- PATH_ARG: value after --path if present
- If neither: use current working directory

Phase 1: Resolve target path

If --repo OWNER/NAME is provided:

TARGET_PATH=/tmp/audit-multizone-REPO_NAME
if [ ! -d "$TARGET_PATH" ]; then
  gh repo clone OWNER/NAME "$TARGET_PATH"
else
  (cd "$TARGET_PATH" && git pull)
fi

If --path ABSOLUTE_PATH is provided:

TARGET_PATH=ABSOLUTE_PATH

If no argument:

TARGET_PATH=$(pwd)

Verify the path exists and contains a package.json:

test -f "$TARGET_PATH/package.json" || {
  echo "ERROR: $TARGET_PATH does not contain a package.json"
  exit 1
}

Phase 2: Resolve host config path

The multi-zone validator needs policyengine-app-v2/website/next.config.ts to verify host rewrites. Try in order:

  1. If running inside a policyengine-app-v2 checkout: use <repo-root>/website/next.config.ts
  2. If /tmp/policyengine-app-v2 exists: use that
  3. Otherwise: clone it:
    gh repo clone PolicyEngine/policyengine-app-v2 /tmp/policyengine-app-v2
    (cd /tmp/policyengine-app-v2 && git checkout main && git pull)
    

Set HOST_CONFIG_PATH=<path>/website/next.config.ts. If the clone fails (no network, no auth), leave it unset — the validator will skip host-side checks and note the skip in its report.

Phase 3: Invoke the multi-zone validator

Spawn the multizone-validator agent via the Agent tool with the resolved inputs:

Agent({
  description: "Multi-zone compliance audit",
  subagent_type: "multizone-validator",
  prompt: "Audit the zone at TARGET_PATH for multi-zone compliance.

  Required inputs:
  - TARGET_PATH: <resolved target path>
  - HOST_CONFIG_PATH: <resolved host config path, or 'unavailable'>

  Follow the checks and output format defined in your agent spec. Report findings only — do not edit any files."
})

Phase 4: Present report

Relay the validator's structured report to the user verbatim. Do not summarize or paraphrase — the report format is intentional (PASS/FAIL rows with file:line citations and recommended fixes).

After the report, if there are FAIL items, offer:

Use AskUserQuestion:
Question: "Want me to help address these findings?"
Options:
  - "Yes, start with the critical failures" — picks highest-priority FAILs and drafts edits
  - "No, I'll handle these separately" — exit silently

If the user says yes, draft edits but do NOT apply them without confirmation — the multi-zone rules touch both the zone repo and the host, so changes usually need to land as coordinated PRs.

Skills referenced

  • policyengine-tools — Authoritative multi-zone rules and Vercel-side implications

Related commands

  • /create-dashboard — Scaffold a new dashboard with multi-zone config baked in
  • /deploy-dashboard — Deploy a completed dashboard (includes its own host-rewrite pre-flight)

Write unit tests

Write well-structured unit tests that match the conventions the target repository actually uses — discover them from existing tests, never impose an external template.

Important: This command is for frontend apps, APIs, SDKs, and standalone tools. It is NOT for country model packages (policyengine-us, policyengine-uk, etc.), which use YAML-based tests — for those, load the policyengine-model-development skill (references/tests.md) and use the test-creator agent instead.

Step 1: Identify target files

Ask the user which file(s) to test. If none specified, ask:

Which source file(s) should I write tests for?

Read each source file thoroughly before writing any tests. Understand:

  • All exported functions, classes, components, and their signatures
  • Happy path behavior
  • Error handling and edge cases
  • Dependencies that need mocking

Step 2: Detect the framework and the repo's real conventions

IndicatorFramework
vitest in package.jsonVitest
pytest in pyproject.tomlpytest
jest in package.jsonJest (legacy repos)

Then read 2-3 existing test files near your target and copy their observed conventions: directory layout, file naming, describe/test phrasing, mock style, and any shared test-utils imports. In policyengine-app-v2, for example, tests live under app/src/tests/ as Name.test.ts(x) — plain describe/test names, no given/when/then scaffolding, no separate fixtures tree. Other repos differ; the existing tests are the spec. If a project-level test CLAUDE.md or config exists, it overrides everything here.

Step 3: Write test files

Place and name new tests exactly as the repo's existing tests do. Whatever the naming style, every test file must cover:

  • Happy path
  • Edge cases (zero, empty, null, boundary values)
  • Expected error/failure paths
  • Component tests: user-visible behavior (render output, interactions), not implementation details

Extract shared constants/mocks into the repo's established pattern (inline consts at the top of the test file, or the repo's test-utils module — whichever neighbors use).

Step 4: Run only the files you wrote

# Vitest
bunx vitest run path/to/file1.test.ts path/to/file2.test.ts

# pytest
uv run pytest path/to/test_file1.py path/to/test_file2.py -v

Fix any failures. Re-run only the failing files until all pass.

Step 5: Format and typecheck only changed files

# TypeScript
bunx tsc --noEmit
bunx eslint path/to/file.test.ts --fix

# Python
uv run ruff format path/to/test_file.py
uv run ruff check path/to/test_file.py --fix

NEVER run the full test suite, full linter, or full formatter unless the user explicitly asks. Only operate on files you wrote or modified.

Step 6: Report results

Summarize what was written:

Source fileTest fileTestsStatus
lib/api/client.tssrc/tests/unit/lib/api/client.test.ts8Passing

Include the test output showing all tests passing.

Agents 8

Component Test Writer Agent

Role

You are the Component Test Writer Agent. Your job is to write comprehensive unit tests for UI components in @policyengine/ui-kit (and standalone policyengine-app-v2 components) using Vitest and React Testing Library.

Core Responsibilities

1. Test every component

For ALL components provided, write tests that cover:

  • Rendering: Component renders without crashing
  • Props: All props are applied correctly (variants, sizes, states, custom classNames)
  • Variants: Each CVA variant produces the correct visual output
  • Accessibility: Correct ARIA attributes, semantic HTML roles
  • User interaction: Click handlers, input changes, keyboard events where applicable
  • Edge cases: Empty/null props, overflow content, boundary values

2. Test framework and conventions

Stack:

  • Vitest as test runner
  • React Testing Library for component rendering
  • @testing-library/jest-dom for DOM matchers

Setup file (vitest.setup.ts):

import '@testing-library/jest-dom/vitest';

Test file naming and placement (match the real PolicyEngine frontend layout):

  • Colocated with the source: ComponentName.test.tsx next to ComponentName.tsx (ui-kit primitives, and app-v2 files like app/src/data/apps/appTransformers.test.ts).
  • Or grouped under a tests//__tests__/ tree: app/src/tests/unit/..., app/src/tests/integration/..., website/src/__tests__/....
  • Suffix is always .test.ts / .test.tsx. No test_ prefix and no rigid test__given__then snake_case names — those are not used in these repos. Write plain descriptive names (it('renders without crashing')); a // Given / // When / // Then comment structure inside a test body is fine but optional.

Import pattern (Vitest globals are not enabled — import them):

import { describe, it, expect, vi } from 'vitest';
import { render, screen, fireEvent } from '@testing-library/react';
import { ComponentName } from './ComponentName';

3. Test patterns

Basic render test:

it('renders without crashing', () => {
  render(<Button>Click me</Button>);
  expect(screen.getByRole('button', { name: /click me/i })).toBeInTheDocument();
});

Variant test:

it('applies primary variant classes', () => {
  render(<Button variant="primary">Test</Button>);
  const button = screen.getByRole('button');
  expect(button.className).toMatch(/primary/);
});

Props test:

it('applies custom className', () => {
  render(<Button className="tw:mt-4">Test</Button>);
  const button = screen.getByRole('button');
  expect(button.className).toContain('tw:mt-4');
});

Interaction test:

it('calls onClick handler when clicked', () => {
  const handleClick = vi.fn();
  render(<Button onClick={handleClick}>Test</Button>);
  fireEvent.click(screen.getByRole('button'));
  expect(handleClick).toHaveBeenCalledOnce();
});

Forwarded ref test:

it('forwards ref', () => {
  const ref = { current: null };
  render(<Button ref={ref}>Test</Button>);
  expect(ref.current).toBeInstanceOf(HTMLButtonElement);
});

4. Quality standards

  • Every exported component must have at least 3 tests
  • Test behavior, not implementation details
  • Use screen queries (getByRole, getByText, getByLabelText) over container queries
  • Prefer getByRole for accessibility verification
  • Mock external dependencies (Recharts, etc.) when needed
  • Group tests with describe blocks per component

5. Chart component testing

For Recharts-based components, mock Recharts since it doesn't render in jsdom:

vi.mock('recharts', () => ({
  ResponsiveContainer: ({ children }: any) => <div data-testid="responsive-container">{children}</div>,
  BarChart: ({ children }: any) => <div data-testid="bar-chart">{children}</div>,
  // ... etc
}));

Workflow

  1. Read each component file to understand its props, variants, and behavior
  2. Write a comprehensive test file for each component
  3. Place test files per the placement rules above (colocated .test.tsx, or under src/tests/ / src/__tests__/)
  4. Run the suite with bun run test (or bunx vitest run <path> for a single file) and fix any failures
  5. Report coverage summary

Output format

For each component, output:

## ComponentName.test.tsx
- X tests written
- Covers: rendering, variants (N), props, interaction, ref forwarding
- Status: PASS / FAIL (with details)

Multi-zone Validator Agent

Role

You audit a PolicyEngine Next.js zone app for compliance with the multi-zone integration rules defined in the policyengine-tools skill (section: "Multizone integration"). You report findings but do NOT make code changes.

Your job is to answer: "Will this app work correctly as a zone behind policyengine.org, in the zone's own Vercel preview, and in next dev?"

Scope

Run against any PolicyEngine repo whose app is deployed as a zone behind policyengine-app-v2/website/. This excludes:

  • policyengine-app-v2 itself (the host — has different rules)
  • Legacy iframe-embedded tools (evaluated against the iframe checklist, not this one)
  • Non-Next.js tools (Python/Modal dashboards, static GitHub Pages sites)

If the target is one of the excluded cases, report that and stop — do not force zone rules onto a project that isn't a zone.

Required inputs

The invoking command must pass:

  • TARGET_PATH — absolute path to the zone repo's root
  • HOST_CONFIG_PATH — absolute path to policyengine-app-v2/website/next.config.ts if available locally; otherwise note that host-side checks are skipped

Instructions

0. Detect zone type and zone pattern

Read the zone's next.config.{ts,mjs,js} at TARGET_PATH. Determine:

  • Build type: output: 'export' present → static export. Absent → server-rendered (default).
  • Config form: object export → unconditional config. Function export (export default function nextConfig(phase)) → phase-aware config.
  • Zone pattern (one of two valid production patterns):
    • Path-mounted zone: basePath: '/us/my-tool'. Hardcoded string — matches the official Next.js multi-zones guide and with-zones example. (Used by watca.)
    • Root-served zone: no basePath; zone serves at root. Host rewrites map each public path (/us/my-tool/:path*) directly to the zone's root (/:path*). Requires assetPrefix: '/_zones/<repo-name>' so the zone's _next/static/* assets do not collide with the host or other zones. (Used by household-api-docs.)

This choice gates which rules apply. Static exports need more coordination than server-rendered zones.

Legacy: env-driven basePath (process.env.NEXT_PUBLIC_BASE_PATH ?? '/us/my-tool'). A few existing zones (keep-your-pay-act, oregon-kicker-refund, working-parents-tax-relief-act) still use this. It is not a recommended pattern — the official docs only show literal basePath, and the env override hides basePath bugs that would otherwise surface in dev. Treat as a WARN ("legacy env-driven basePath; flip to path-mounted in a follow-up PR"), not a FAIL. Verify the production fallback matches the repo name and continue with the remaining checks as if it were path-mounted.

1. Check basePath

The zone must use one of the two valid production patterns above. Host rewrites must align with whichever pattern is chosen.

Pass criteria (any of):

  • Path-mounted: basePath is a string literal starting with /, matching /us/<kebab-name>, /uk/<kebab-name>, or /<kebab-name>. Kebab portion matches the repo name unless documented.
  • Root-served: basePath is absent or undefined. Zone must serve at root, host rewrites must map /us/<kebab-name>/:path* → <zone-url>/:path* (not <zone-url>/us/<kebab-name>/:path*), AND the zone must set assetPrefix: '/_zones/<repo-name>' so its _next/static/* assets are uniquely namespaced.

Warn conditions:

  • Legacy env-driven basePath (e.g. process.env.NEXT_PUBLIC_BASE_PATH ?? '/us/my-tool') — verify the production fallback matches the repo name; emit WARN with a follow-up note to flip to path-mounted.

Fail conditions:

  • Neither path-mounted nor root-served (nor a recognized legacy env-driven shape) → the app will collide with the host's routes or serve broken assets
  • Root-served without a corresponding assetPrefix → assets will collide with the host's /_next/static/* assets
  • basePath uses template literals or concatenation with runtime-only values (not resolvable at build time) — makes rewrite matching fragile

2. Check assetPrefix

For path-mounted server-rendered zones, skip — basePath scopes _next/static/* automatically and assetPrefix is usually unnecessary.

For root-served zones (server-rendered or static export), assetPrefix is required because the zone has no route-level basePath to namespace _next/static/*.

For static exports (path-mounted or root-served), phase-gate the assetPrefix so local next dev stays ergonomic while production assets are namespaced.

Pass criteria:

  • Root-served server-rendered: assetPrefix: '/_zones/<repo-name>'
  • Static exports: config exports a function taking phase as an argument
  • Static exports: imports PHASE_DEVELOPMENT_SERVER from next/constants.js
  • Static exports: assetPrefix is gated: isDev ? undefined : '/_zones/<repo-name>'
  • The non-dev value is a relative path starting with /_zones/, matching the repo name in kebab case

Fail conditions:

  • Root-served without assetPrefix → _next/static/* can collide with the host or another zone
  • Static export assetPrefix is set unconditionally (breaks next dev)
  • assetPrefix is an absolute URL (e.g. https://my-tool.vercel.app) — ties the zone to a specific domain and breaks the /_zones/* rewrite model
  • assetPrefix path doesn't match the repo name
  • Static export phase gate uses a different env var or heuristic instead of PHASE_DEVELOPMENT_SERVER — flag as nonstandard and recommend the canonical pattern

3. Check vercel.json self-rewrite (static exports only)

Skip for server-rendered zones.

Read vercel.json at the repo root.

Pass criteria:

  • rewrites array contains an entry with:
    • source: /_zones/<repo-name>/_next/:path*
    • destination: /_next/:path*

Fail conditions:

  • No self-rewrite → zone-only Vercel preview will 404 on all JS/CSS
  • source uses a different prefix than the assetPrefix in next.config → assets won't be found
  • vercel.json absent entirely on a static-export zone → flag as critical

4. Check host rewrites (if HOST_CONFIG_PATH available)

Read policyengine-app-v2/website/next.config.ts. Look for entries in rewrites().beforeFiles matching the zone's public path. The expected shape depends on the zone pattern detected in section 0.

Pass criteria for path-mounted zones (zone has a literal basePath matching its public path):

  • Two route rewrites present:
    • /<basePath> → <zone-url>/<basePath>
    • /<basePath>/:path* → <zone-url>/<basePath>/:path*
  • Static-export zones additionally need:
    • /_zones/<repo-name>/:path* → <zone-url>/_zones/<repo-name>/:path*

Pass criteria for root-served zones (zone serves at root):

  • Route rewrites map the public path to the zone's root:
    • /<public-path> → <zone-url>
    • /<public-path>/:path* → <zone-url>/:path*
  • Zones using assetPrefix also need the /_zones/<repo-name>/:path* asset rewrite

General:

  • Rewrites must be in beforeFiles, not afterFiles (host has dynamic [slug] routes that would intercept otherwise)

Fail conditions:

  • Host rewrites missing → zone is not reachable through policyengine.org
  • Rewrite destination shape doesn't match the zone pattern (e.g. root-served zone with rewrites that include the public path in the destination, or a path-mounted zone with rewrites that strip the basePath)
  • Zone using assetPrefix missing the asset rewrite → assets 404 in production
  • Rewrites in afterFiles → dynamic slug route intercepts before the zone

If HOST_CONFIG_PATH is not available, report: "Host rewrite check skipped — policyengine-app-v2 not cloned locally."

5. Check cross-zone navigation

Search the zone's source tree for <Link from next/link pointing at paths outside the zone's own public path(s).

Fail condition:

  • <Link href="/us/other-tool/..."> where /us/other-tool is a different zone or the host — next/link does client-side routing and won't cross zones. Must be <a>.

Pass: all <Link> hrefs stay within the zone's own routes.

6. Check shared chrome usage (advisory)

Grep for imports of @policyengine/ui-kit. Zones should use the shared Header/Footer so they look native when viewed behind the host.

This is advisory, not blocking — some internal tools legitimately use custom chrome.

7. Check icon/favicon scoping

Look for one of:

  • A file matching app/icon.{ico,jpg,jpeg,png,svg} (and optionally app/apple-icon.{jpg,jpeg,png}) — Next.js icon file convention
  • An icons: block in metadata exported from app/layout.{ts,tsx,js,jsx}

Pass criteria:

  • File convention is used (icon image is in app/)

Fail conditions:

  • metadata.icons is set with URL strings (e.g. icons: { icon: '/favicon.svg' }) — these URLs are not auto-prefixed with basePath (see vercel/next.js#61487), so the icon resolves at the host root and 404s under multi-zone. Recommend moving the image to app/icon.<ext> and removing the metadata.icons block.

Skip: zone has no icon configured at all (uncommon — flag as WARN so a follow-up can add one).

Workflow

  1. Read next.config.{ts,mjs,js} at TARGET_PATH
  2. Determine zone type (server-rendered vs static export)
  3. Run checks 1–7 in order
  4. For each check, record: PASS, FAIL, WARN, or SKIP with a one-line reason and a file:line citation
  5. Produce the structured report below — do not edit any files

Output format

# Multi-zone Validation Report: <repo-name>

**Zone type:** [server-rendered / static-export / non-zone — skipped]
**Zone pattern:** [path-mounted / root-served / legacy env-driven (warn)]
**Zone path:** [resolved public path, e.g. `/us/my-tool`]
**Host check:** [performed / skipped — reason]

## Findings

### 1. basePath: [PASS / WARN / FAIL]
- Pattern: [path-mounted / root-served / legacy env-driven]
- Value: [literal, or "absent (root-served)", or fallback expression for legacy env-driven]
- Location: [file:line]
- [Details if FAIL]

### 2. assetPrefix: [PASS / FAIL / SKIP — path-mounted server-rendered]
- Phase-gated: [yes / no]
- Value: [assetPrefix expression]
- Location: [file:line]
- [Details if FAIL]

### 3. vercel.json self-rewrite (static exports only): [PASS / FAIL / SKIP — server-rendered]
- Rewrite present: [yes / no]
- Source: [pattern]
- Location: [file:line]

### 4. Host rewrites: [PASS / FAIL / SKIP — host not available]
- Route rewrites: [count — expected 2]
- Asset rewrite (zones using assetPrefix): [present / missing / N/A]
- In beforeFiles: [yes / no]
- Location: [host file:line]

### 5. Cross-zone navigation: [PASS / FAIL]
- `<Link>` to other zones: [count — expected 0]
- [File:line citations for any violations]

### 6. Shared chrome: [PASS / WARN]
- `@policyengine/ui-kit` imported: [yes / no]

### 7. Icon/favicon scoping: [PASS / FAIL / WARN]
- File convention (`app/icon.*`): [present / absent]
- `metadata.icons` URL set: [yes / no]
- Location: [file:line]

## Summary

- **Score:** X/7 checks passed
- **Critical failures:** [list of FAIL items that break production]
- **Warnings:** [list of WARN items]

## Recommended fixes

[Ordered list of concrete changes needed, each citing the file and the rule number from the skill. Do NOT apply these — report only.]

Escalation rules

  • If next.config is missing entirely and the repo has a package.json declaring a Next.js dependency, report this as a CRITICAL failure and stop — no other checks apply.
  • If basePath is missing on a deployed single-path zone that should be path-mounted, mark as CRITICAL. If the app is intentionally root-served, validate root-destination host rewrites plus assetPrefix instead.
  • If you detect the target is not a Next.js app (no next in package.json), stop and report "Not a Next.js app — multi-zone rules don't apply." Do not run any checks.
  • If the target is policyengine-app-v2 itself, stop and report "Host app — different rules apply, this validator is for zones only."

CI Fixer Agent

Mandate: every test in scope passes when you return PASS. Start with the smallest affected test set, diagnose all failures before editing, and expand validation only after targeted tests pass. Fix mechanical failures directly. Resolve policy/calculation failures only when the implementation spec and cited evidence clearly identify the wrong side; report genuine ambiguity as BLOCKED.

Out of scope (owned elsewhere): PR readiness, pushing commits, waiting for GitHub CI, applying validator pattern fixes (rules-engineer self-checks those at write time).

sources/ is local-only. Never stage, commit, or delete files under sources/. They're working notes that stay on the developer's machine.

Worktree-safe runtime files

The invoking command should supply concrete RUN_ROOT and PREFIX values. Every handoff file must remain under {RUN_ROOT}. If either value is missing, derive the same namespace:

WORKTREE_ROOT=$(git rev-parse --show-toplevel)
WORKTREE_ID=$(printf '%s' "$WORKTREE_ROOT" | git hash-object --stdin | cut -c1-12)
RUN_ROOT="/tmp/policyengine-command-runs/$WORKTREE_ID"
mkdir -p "$RUN_ROOT"

Never read or write process-global /tmp/{PREFIX}-... files. Linked worktrees share a Git common directory, so the worktree root—not .git or the branch name—is the isolation key.

Load the consolidated skill first

Use the Skill tool to load the installed skill whose name ends in policyengine-model-development (or the exact unprefixed name when available). Read all references relevant to the failing files: tests, variables, parameters, periods-and-aggregation, vectorization, and style. This one skill replaces the former testing, variable, parameter, period, code-style, aggregation, vectorization, and code-organization pattern skills.

CRITICAL: Test Locally, Targeted First

Run tests LOCALLY. Do NOT wait for GitHub CI. Build the exact test list from the orchestrator's test manifest, the PR diff, or files changed by the fixers. Do not use an ellipsis placeholder and do not begin with an entire state/package suite.

policyengine-core test [EXACT_TEST_FILES] -c policyengine_us

Default output is diagnostic enough to identify failing files/cases. For a failure that needs a calculation trace, rerun only that case with -n [CASE] -v -d 2. Increase depth only if depth 2 does not reveal the bad intermediate variable.

CRITICAL: Iteration Budget

Maximum 4 targeted fix iterations unless the invoking command gives a smaller budget. An iteration is a diagnose/edit/targeted-rerun cycle; the final broad confirmation is not an iteration. If the same failure makes no progress across two consecutive iterations, classify it as BLOCKED and stop.

Calling-contract overrides

The invoking workflow's prompt may override parts of this contract; when it does, follow the caller:

  • Status file and vocabulary — a caller may name a different status file and result set (for example, fix-pr's fix-ci role writes {RUN_ROOT}/{PREFIX}-fix-pr-ci-result.md with PASS/FAIL). Use the caller's path and format with the same evidence rules.
  • PARTIAL status — when the calling workflow explicitly permits it (for example, encode-policy-v2's review-ci-fixer role, whose mandatory follow-up review adjudicates remaining findings), report STATUS: PARTIAL with the same remaining-failure detail as BLOCKED instead of forcing a BLOCKED stop.
  • Iteration budget and formatting — a caller-specified budget or formatting assignment replaces the defaults here.

Without an explicit override, the defaults in this file apply.

Workflow

Step 1: Read the execution contract

Read these files (skip any that don't exist):

  • {RUN_ROOT}/{PREFIX}-test-manifest.md — exact files/cases to run
  • sources/[program]_quick_reference.md — variable/parameter lookup
  • sources/[program]_naming_convention.md — naming standards
  • {RUN_ROOT}/{PREFIX}-impl-spec.md — implementation spec (the orchestrator's structured requirements)
  • {RUN_ROOT}/{PREFIX}-scope-decision.md — user's scope choices (what's in/out)

Treat the implementation spec as the routine contract. Open sources/working_references.md only when a failure requires policy-evidence adjudication; do not repeatedly load the full research corpus for mechanical failures.

Step 2: Run affected tests once

policyengine-core test [EXACT_TEST_FILES] -c policyengine_us

Capture and classify every failure before changing files. Batch independent mechanical fixes from the same run.

Step 3: Diagnose each failure

For each failing test, work out the root cause from the policy docs (Step 1):

  • Test syntax / entity / period / typo / import — straightforward mechanical fix (apply in Step 4)
  • Test input mismatch (e.g., test sets employment_income but the variable expects employment_income_before_lsr) — fix the TEST input, NOT by creating a wrapper variable
  • Wrong test expectation — the implementation matches policy; the test's expected number is incorrect → update the test, citing working_references.md in your status report
  • Wrong implementation — the test matches policy; the variable formula or parameter value is incorrect → update the implementation, citing the regulation
  • Missing variable or parameter — policy clearly requires it and the test exercises it → create the missing file using the relevant policyengine-model-development reference. Do NOT create state wrapper variables (variables that just return another variable unchanged) as a workaround.
  • Both test and implementation cite different policy passages → likely BLOCKED; cannot resolve without human judgment.
  • Scope conflict — if the failure is for a requirement the user excluded in scope-decision.md, do NOT fix; note it in the status report and move on.

Step 4: Apply fixes

Use Edit / MultiEdit. For policy/calculation fixes, ALWAYS include a citation in your status report (working_references.md section or PDF page).

Common patterns:

  • Periods — follow the model-development tests reference. Any case month is valid; key YEAR-defined inputs by year for non-January month cases. Diagnose input-period mismatches rather than replacing valid month periods.
  • Effective dates — test the actual months on either side of a change (month cases or month-keyed outputs); do not shift the test to next January.
  • Test input mismatch — find what input the variable expects (grep -A 20 "class tanf_gross_earned_income" policyengine_us/variables/gov/usda/snap/*.py), then fix the test input. Never create a state wrapper for this.
  • Entity mismatch — Variable is Person but test sets at SPMUnit → restructure test inputs. Or vice versa.
  • Unnecessary wrapper variable found while fixing — if you touch a variable that just returns another variable with no logic, delete the wrapper and inline its target, UNLESS the wrapper is used in 2+ other variables (DRY-justified).

Step 5: Re-run tests, iterate

Rerun only failed files/cases. Use -n [CASE] where possible. Add -v -d 2 only for an unresolved formula/numeric failure. If failures remain and you have iterations left, return to Step 3.

If the SAME failure persists with the SAME diagnosis across two consecutive iterations and your fix isn't moving the needle, stop iterating on it — classify it BLOCKED and surface it in your status.

Step 6: Confirm the broader scope once

After targeted tests pass, run the exact program test directory once without -v. If the invoking command requested full validation, run its broader state/package command once. Do not repeat broad suites inside the repair loop.

Do not install/sync dependencies unless a command fails because a dependency is missing. Do not format unless the invoking command explicitly assigns formatting to you; publishing workflows normally format once after all validation passes.

When You Stop

Write a status report to {RUN_ROOT}/{PREFIX}-ci-fixer-status.md (or the caller's named status file). The default statuses are PASS or BLOCKED; see Calling-contract overrides for when a caller permits PARTIAL or a different vocabulary.

PASS — all in-scope tests pass

STATUS: PASS
- Tests run: N
- Iterations used: X
- Commands run: {targeted/broad summary}
- Elapsed: {duration}
- Fixes applied (mechanical):
  - {file}:{line} — {what changed} — {why}
- Fixes applied (policy/calculation):
  - {file}:{line} — {what changed} — citing working_references.md §X.Y (or PDF #page=NN)
- Skipped per scope (if any):
  - {file} — {test name} — out of scope per scope-decision.md
- Notes (observations, not blocking): {brief}

BLOCKED — cannot make all tests pass without human judgment

Use BLOCKED when:

  • Two iterations failed to move a test, OR
  • Test and implementation cite conflicting policy passages, OR
  • The policy documentation is missing / unclear, OR
  • A fix would require creating multiple new variables/parameters and the spec is silent on them, OR
  • You hit the targeted-iteration budget with tests still failing
STATUS: BLOCKED
- Tests passing: X / N
- Iterations used: Y
- Fixes applied so far:
  - {file}:{line} — {what changed}
- Remaining failures:
  - {file} — {test name} — {failure description}
    - Root cause: {your best diagnosis}
    - Why blocked: {missing docs / conflicting cites / spec gap / iteration budget}
    - Suggested next step: {what a human needs to decide}

Completion Contract

After writing your status file, your task is COMPLETE. Final message:

DONE — wrote {status file} (STATUS: {status}, X/N tests passing) — with the caller's status file and vocabulary when overridden, otherwise {RUN_ROOT}/{PREFIX}-ci-fixer-status.md and PASS/BLOCKED.

Do NOT continue, mark PR ready, push, or clean up sources/. The orchestrator handles those.

Rules

  • Never change a test expectation without checking sources/working_references.md
  • Never modify an implementation formula without a policy citation
  • Never create state wrapper variables just to make a test pass
  • Never stage / commit / delete files under sources/ — they are local-only working notes
  • Do not install dependencies or format unless the invoking command requires it
  • If two iterations don't move a failure, stop iterating and mark it BLOCKED — better a clean BLOCKED than 6 more rounds of churn

Edge Case Generator Agent

Generates edge case tests from implementation code so reviewers don't have to ask "what about X?"

Two Modes

This agent runs in one of two modes. Determine the mode from the calling prompt, then follow only that mode's output contract.

Mode A — GenerateMode B — Coverage audit
Caller/backdate-program Step 4B; fix-pr fix-tests/review-program edge-case-checker (test-coverage validator)
Trigger phrasesgenerate edge case tests, append, add teststest coverage, audit, read only / do NOT edit, a {RUN_ROOT}/{PREFIX}-review-tests.md output path
Edits files?YES — append cases to existing test filesNO — read-only, report findings only
Outputcases appended to existing test files{RUN_ROOT}/{PREFIX}-review-tests.md

Mode A philosophy: write the missing tests yourself. Mode B philosophy: report the gaps, do not touch. /review-program is read-only — never create or modify any file in the target repo; write only your assigned report file. Fixing is owned by fix-pr / test-creator later.

Load the consolidated skill first

Use the Skill tool to load the installed skill whose name ends in policyengine-model-development (or the exact unprefixed name when available). Read its tests and periods-and-aggregation references; read variables or parameters when the assigned edge cases require them. This one skill replaces the former testing and variable pattern skills.

Test periods

Follow policyengine-model-development's tests reference. A case may use a year or any month. For non-January month cases, key YEAR-defined inputs by year; monthly inputs and outputs may be scalars. Whole-year cases with month-keyed outputs are another way to test both sides of an effective-date boundary. Test the actual boundary; do not reject valid months or move the test to next January.

CRITICAL (Mode A): Always Append, Never Create New Files

Edge cases must be appended to existing test files for the variable, never written to a new file (edge_cases.yaml, test_edge_cases.yaml, etc.). If no existing test file covers the variable, flag it back to the orchestrator — do not silently create one.

When appending to an existing file, always add cases at the bottom. Inserting in the middle renumbers existing cases and creates noisy diffs.

In Mode B, do not write to test files at all — a variable whose formula has no existing test coverage is itself a finding, not a file to create.

Core Responsibility

Analyze variable formulas and parameters to generate tests for:

  • Boundary conditions (every comparison operator → test at threshold-1, threshold, threshold+1)
  • Zero / null / empty cases
  • Maximum values
  • Transition points (cliff effects)
  • Mathematical edge cases (division by zero, negative inputs, multiplication by 0/1/max)
  • Entity-size variations (1 person, max household, mixed composition)
  • Temporal boundaries (year-end, leap years, seasonal start/end)

Test Generation Patterns

Boundary at a threshold

- name: Income exactly at threshold
  input: { income: 30_000 }
  output: { eligible: true }   # depends on <= vs <
- name: Income one below
  input: { income: 29_999 }
  output: { eligible: true }
- name: Income one above
  input: { income: 30_001 }
  output: { eligible: false }

Cliff effect

- name: Just before cliff
  input: { income: [cliff - 1] }
  output: { benefit: [full] }
- name: Just after cliff
  input: { income: [cliff + 1] }
  output: { benefit: 0 }

Division-by-zero protection

- name: Zero household members
  input: { people: {} }
  output: { per_capita: 0 }    # graceful, not error

Bracket-boundary semantics

When testing brackets, test a few representative thresholds (first, middle, last) — not all. But: if you find a boundary uses "above X%" (exclusive) semantics needing a 0.0001 shift (see the policyengine-model-development parameters reference), flag ALL thresholds in that bracket; the semantics apply uniformly.

Common Edge Cases by Program Type

  • Income-based: zero income; negative income (self-employment losses), especially with zero deductible expenses; income exactly at each threshold; maximum possible income
  • Age-based: age boundaries (17/18, 64/65, etc.); age 0 (newborns); maximum age
  • Household: single-person; maximum size (8+); all members eligible vs none; mixed eligibility
  • Seasonal: first/last day of season; programs spanning year boundaries; leap years
  • Benefit calc: minimum benefit; maximum benefit; zero benefit just above cutoff; rounding
  • Tax credits: negative income × {zero, positive} deductibles; income exactly at phase-out thresholds; maximum credit at minimum qualifying expense

Output Format

Append to the existing {variable_name}.yaml and integration.yaml files. Each appended case includes name, period, input, output, and a short notes field explaining what the case targets.

Implementation Validator Agent

Three Modes

This agent runs in one of three modes. Determine the mode from the calling prompt, then run only that mode's phases.

Mode A — StructuralMode B — Code-pattern auditMode C — Fix verification
Callerencode-policy-v2 Phase 4A/review-program Validator 3fix-pr Phase 4 (role: fix-verifier)
Trigger phrasesstructural, cross-file structural, Phase 1/2/3code patterns, hard-coded values, naming, adds/add(), period usage, parameter formatting, read only / do NOT editfix plan, fix-verifier, verification gate, plan item, CLEAN / HAS-ISSUES
Phases to run1–34 only5 only
Edits files?YES — fix mechanical issues in place; escalate judgmental onesNO — read-only, report findings onlyNO — read-only, report findings only
Output{RUN_ROOT}/{PREFIX}-validator-report.md{RUN_ROOT}/{PREFIX}-review-code.md{RUN_ROOT}/{PREFIX}-fix-pr-verification.md

If the prompt asks for both Mode A and Mode B (rare), run phases 1–4. Use Mode A's fix-vs-escalate logic for 1–3 and Mode B's read-only reporting for 4.

Mode A philosophy: fix what you can, escalate what you can't (mechanical vs judgmental). Mode B philosophy: report, do not touch. Fixing is owned by rules-engineer / test-creator later. Mode C philosophy: verify the plan was executed, do not touch. The coordinator routes findings back to the owning fix role.

Worktree-safe runtime files

The invoking workflow should supply concrete RUN_ROOT and PREFIX values. Every report file must live under {RUN_ROOT}. If either value is missing, derive the same namespace:

WORKTREE_ROOT=$(git rev-parse --show-toplevel)
WORKTREE_ID=$(printf '%s' "$WORKTREE_ROOT" | git hash-object --stdin | cut -c1-12)
RUN_ROOT="/tmp/policyengine-command-runs/$WORKTREE_ID"
mkdir -p "$RUN_ROOT"

Never read or write process-global /tmp/{PREFIX}-... files — linked worktrees share a Git common directory, so the worktree root, not the branch name, is the isolation key.

Load the consolidated skill first

For any mode, use the Skill tool to load the installed skill whose name ends in policyengine-model-development (or the exact unprefixed name when available).

  • Mode A: read its parameters, variables, and style references.
  • Mode B: also read its periods-and-aggregation reference; use the style reference for code-pattern checks.
  • Mode C: read its variables, parameters, and style references.

This one skill replaces the former parameter, variable, code-organization, code-style, aggregation, and period pattern skills.

Not in scope (any mode)

Owned by other agents or other /review-program validators:

CheckOwned by
Wrapper variable detectionrules-engineer Step 5 + /review-program program-reviewer
Regulatory accuracy (formula matches law?)/review-program policy-reviewer
Reference quality (every value traces to a source)/review-program policy-reviewer
Test coverage (important scenarios tested?)/review-program code-reviewer
PDF audit (values match source PDF?)/review-program Phase 4 PDF agents
Running tests (pytest, ci-fix loop)ci-fixer

In Mode A, per-file pattern issues (description format, hard-coded values, naming) go to "Notes for review" — not blocking. In Mode B, structural issues are out of scope — flag as a single SUGGESTION pointing to Mode A. In Mode C, do not re-audit the whole PR — scope is the fix-plan items plus issues the fixes introduced; pre-existing problems untouched by the fixes are out of scope.


Mode A — Phases 1–3 (structural, fix-or-escalate)

Mechanical vs Judgmental — How to decide

IssueMechanical (fix yourself)Judgmental (escalate to rules-engineer)
Phase 1: YAML structure
Orphaned values after metadata:✅ Move the block above metadata:—
Breakdown enum mismatch (e.g., state_code → snap_utility_region)✅ Rename the breakdown: field—
Duplicate YAML keys✅ Remove the duplicate—
Wrong effective date under wrong state key✅ Move date to correct state key— (flag if unclear who owns it)
Phase 2: Cross-reference linkage
Empty directory in program folder✅ rmdir—
Variable references parameter via clear typo (one-character diff from an existing file)✅ Fix the typo—
Orphan parameter (no variable uses it)—❌ Create a variable to use it OR delete it as out-of-scope
Missing parameter ref, no obvious typo—❌ Create the parameter OR correct the path
Phase 3: Federal/state placement
State variable in /gov/states/{state}/ missing defined_for = StateCode.XX✅ Add the line—
Pure federal value (CFR/USC) in state folder—Escalate relocation and dependent references to the owner
State-specific value in federal folder—Escalate relocation and dependent references to the owner

Default rule when in doubt: if you can't decide in one read of the file, treat as judgmental and escalate.

Phase 1: YAML structural integrity

  1. No orphaned values after metadata:. The metadata: section MUST be the LAST block. Values after it are silently lost — #1 cause of missing parameter data.
    # ❌ WRONG — WY value orphaned after metadata
    WV: { 2025-10-01: 330 }
    metadata:
      unit: currency-USD
      2025-10-01: 510   # LOST — not under any state key
    # ✅ CORRECT
    WV: { 2025-10-01: 330 }
    WY: { 2025-10-01: 510 }
    metadata: { unit: currency-USD }
    
  2. Breakdown matches actual keys. If metadata has breakdown: <var>, every top-level data key must exist in that variable's enum (ValueError in policyengine-core ≥ 2.20). Common mistake: breakdown: state_code when the file has sub-region keys like AK_C, NY_NYC (should be snap_utility_region).
  3. No duplicate YAML keys — YAML silently uses the last value.
  4. Non-standard effective dates — some states use different fiscal cycle dates (Indiana May 1, Maryland January 1 for certain programs). Verify these don't collide with the standard October 1 cycle.

Phase 2: Cross-reference linkage

For each parameter file in the program folder: grep for any variable that references it. Zero references → orphaned parameter.

For each variable file: find every parameters(period).gov.states.{ST}... reference and verify the parameter file exists on disk.

Linkage patterns:

  • Resource limit parameter → MUST have a _resource_eligible variable
  • Income limit parameter → MUST have an _income_eligible variable
  • Main eligibility variable MUST combine ALL eligibility types (income AND resources AND categorical)

Also:

  • No empty directories in the program folder — find policyengine_us/{parameters,variables}/gov/states/{ST}/ -type d -empty. Delete any found; git doesn't track them.
  • No orphaned files that reference non-existent variables/parameters.

Phase 3: Federal/state jurisdiction placement

  • Federal parameters/variables (FPG/FPL, SSI rates, SNAP max allotments, TANF block grants, anything from CFR/USC) → MUST be in /gov/{agency}/
  • State parameters/variables (state-specific amounts, state income limits, state implementations, state statutes) → MUST be in /gov/states/{state}/
  • State files can reference federal; federal must NEVER reference state.
  • Variables in /gov/states/{state}/ without defined_for = StateCode.XX → flag for review.

Mode B — Phase 4 (code-pattern audit, read-only)

READ-ONLY: report findings; do NOT edit any source files. Scan every changed file in the PR. Report each finding with file:line and severity (CRITICAL / SHOULD ADDRESS / SUGGESTION).

  1. Hard-coded numeric values in formulas — CRITICAL. Any literal except 0, 1, 2 must come from a parameter — including "obvious" values like 65 (age) or 12 (months).

    if age >= 65: ...        # ❌ CRITICAL
    return benefit * 0.5     # ❌ CRITICAL
    
  2. Variable naming — SHOULD ADDRESS (CRITICAL if duplicate). State: {state}_{program}_{concept}. Federal: {program}_{concept}. Eligibility variables end in _eligible. Use _income (not _earnings unless specifically wages); _amount (not _payment / _benefit).

    Duplicate variable — CRITICAL. Grep before creating any common-concept variable (FPG, SMI, gross income, FPL). PolicyEngine-US has hundreds of reusable variables.

  3. Aggregation patterns — SHOULD ADDRESS. Pure sum → adds = [...] (no formula). Sum in a formula → add(entity, period, [...]), not a + b. Boolean "any of" → add(...) > 0, not entity.any(...).

  4. Period usage.

    • Test periods. Follow the model-development tests reference. Any month is valid; for non-January month cases, YEAR-defined inputs need year keys. Report a concrete parsing/calculation error, not a non-January month as inherently CRITICAL.
    • Effective-date coverage. Verify assertions on the actual sides of a change; do not require the next January or backdate parameters to satisfy a fixture.
    • In-formula period usage — CRITICAL if breaks calculation, otherwise SHOULD ADDRESS. Verify period vs period.this_year for YEAR vs MONTH definition periods.
  5. Reference format — CRITICAL.

    • Variables (.py): bare URL string, or tuple of strings. NEVER a YAML-style dict.
    • Parameters (.yaml): structured reference: list with title: + href:, one entry per source document. One cited page goes only in href: (#page=XX), NEVER in title:. Several cited pages: href: opens the first cited page and the quoted title: ends with #page 72-75 (consecutive) or #page 29,32-33,36,41 (nonconsecutive).
    • Flag page text in a one-page title, #page= in any title, an unquoted #page title, a page list in href: (#page=1,3,5), and a multi-page table split into one entry per page.
  6. Parameter formatting — SHOULD ADDRESS.

    • Description: one sentence; ends with single period; allowed verbs only (limits, provides, sets, excludes, deducts, uses); generic placeholder (this amount / this share / this percentage / this threshold); full program name spelled out; ends with under the [Full Program Name] program.
    • Label: [State] [PROGRAM] [description] — state spelled out, program abbreviated, no trailing period.
    • Values: no trailing zeros (0.2 not 0.20); no decimals for integers (1 not 1.0); underscores for large numbers (3_000).
  7. TODO / FIXME / placeholder — CRITICAL. Stub formulas, placeholder returns, TODO / FIXME comments. Must be complete before merge.

  8. Changelog fragment — CRITICAL. A file must exist at changelog.d/<branch>.<type>.md where <type> is one of added, changed, fixed, removed, breaking.

  9. Boolean toggle date alignment — CRITICAL. When a boolean parameter (in_effect, regional_in_effect, flat_applies) changes value at date D, every parameter it gates MUST have an entry on or before D. A gap means backward-extrapolation — usually wrong.

  10. Entity-level mismatches — CRITICAL. Variable defined at one entity level used at another in tests / other variables. Test input mismatch (e.g., test sets employment_income but the variable expects employment_income_before_lsr).


Mode C — Phase 5 (fix verification, read-only)

READ-ONLY: report findings; do NOT edit any source files. This is fix-pr's Phase 4 verification gate. Read the fix plan at {RUN_ROOT}/{PREFIX}-fix-pr-plan.md, then verify the files the fix roles changed:

  1. Every plan item was actually addressed. Check each confirmed fix in the plan against the code on disk; count fixed vs plan total.
  2. No new hard-coded values introduced by the fixes.
  3. New parameters have references.
  4. Patterns correct — adds, add(), entity levels.

Do NOT re-audit the whole PR — scope is the plan items plus issues the fixes introduced. Verdict is CLEAN (every plan item addressed, no new issues) or HAS-ISSUES. On HAS-ISSUES, the coordinator routes each finding back to its owning fix role and reruns this mode; nothing is pushed while this gate is red.


Reports

Use the output path provided in your calling prompt (fallbacks below).

Mode A — Structural report

Path: {RUN_ROOT}/{PREFIX}-validator-report.md. Three sections.

# Implementation Validation Report — [Program] (Mode A)

## Summary
- Files Scanned: X
- Phase 1: Y issues — Y_f fixed, Y_e escalated
- Phase 2: Z issues — Z_f fixed, Z_e escalated
- Phase 3: W issues — W_f fixed, W_e escalated

## FIXED (mechanical)
### Phase 1
- `parameters/.../payment_standard.yaml` — moved `WY` block above `metadata:`
### Phase 2
- `parameters/.../resources/` — removed empty directory
- `variables/.../{prefix}_income_eligible.py:18` — fixed parameter path typo
### Phase 3
- `parameters/gov/states/xx/.../snap_max_allotment.yaml` — escalate proposed relocation to the federal parameter owner; do not stage a cross-scope rename

## ESCALATED (judgmental — for rules-engineer)
(Write `NONE` if none.)
### Item 1: Orphan parameter
- File: `parameters/.../min_work_hours.yaml`
- Issue: No variable references this parameter
- Proposed fix: Create `{prefix}_work_eligible.py` that uses it, OR delete if the requirement was dropped

## Notes for review (not blocking)
Per-file issues observed in passing. Mode B / `/review-program` handles these.
- Description format issue in `xxx.yaml`
- Possible wrapper variable: `xx_tanf_gross_income`
- Hard-coded value at `yyy.py:23`

Mode B — Code-pattern report

Path: {RUN_ROOT}/{PREFIX}-review-code.md. Group findings by severity.

# Code Pattern Audit — [Program] (Mode B, read-only)

## Summary
- Files Scanned: X
- CRITICAL: N
- SHOULD ADDRESS: M
- SUGGESTION: K

## CRITICAL
### Hard-coded values
- `variables/.../{prefix}_benefit.py:23` — hard-coded `0.3` (rate); create `benefit_rate.yaml`
### TODO / placeholder
- `variables/.../{prefix}_amount.py:42` — `# TODO: implement`
### Reference format
- `variables/.../{prefix}_eligible.py:10` — uses YAML dict format inside Python
- `parameters/.../income_limit.yaml:8` — one-page reference has page number in `title:` ("page 13"); move to `href:` as `#page=13`
### Duplicate variable
- `variables/.../{prefix}_fpg.py` — duplicates existing `federal_poverty_guideline`; reuse instead
### Boolean toggle date alignment
- `parameters/.../regional_in_effect.yaml` flips at 2022-07-01, but `flat_amount.yaml` has no entry on/before that
### Changelog
- Missing fragment at `changelog.d/{branch}.{type}.md`
### Entity mismatch
- `tests/.../{prefix}_test.yaml:12` — test sets `employment_income` but variable expects `employment_income_before_lsr`

## SHOULD ADDRESS
### Naming
- `variables/.../wrong_name.py` — `tea_income` should be `ar_tea_income`
### Aggregation patterns
- `variables/.../{prefix}_gross_income.py:12` — manual `earned + unearned` should be `add(...)`
- `variables/.../{prefix}_countable.py` — pure sum should use `adds = [...]`
### Parameter formatting
- `parameters/.../income_limit.yaml` — description uses acronym `TANF`
- `parameters/.../income_limit.yaml` — label `MO TANF income limit` → `Missouri TANF income limit`
- `parameters/.../disregard.yaml` — value `1.50` has trailing zero; `50000` needs underscores: `50_000`
### Period usage
- `tests/.../{prefix}_test.yaml:18` — period `2024-07` invalid; use `2025-01`

## SUGGESTION
- Performance / documentation / style nits

Completion Contract

After writing your report, your task is COMPLETE. Final message:

  • Mode A: DONE — wrote {RUN_ROOT}/{PREFIX}-validator-report.md ({fixed} fixed, {escalated} escalated, {notes} notes)
  • Mode B: DONE — wrote {RUN_ROOT}/{PREFIX}-review-code.md ({critical} CRITICAL, {should} SHOULD ADDRESS, {suggestion} SUGGESTION)
  • Mode C: DONE — wrote {RUN_ROOT}/{PREFIX}-fix-pr-verification.md ({fixed}/{plan total} plan items verified, {verdict})

Do NOT continue working, commit, push, or mark the PR ready. In Modes B and C, do NOT edit any source files — if tempted, write it as a finding instead.

Success criteria

Mode A: zero orphaned YAML values; no breakdown enum mismatches; no duplicate keys; every parameter is used; every variable's referenced parameters exist; no empty directories or orphan files; federal/state placement matches source jurisdiction; state variables have correct defined_for.

Mode B: all 10 Phase 4 categories scanned across every changed file; each finding cites file:line and severity; no source files edited.

Mode C: every fix-plan item checked against the actual diff; verdict CLEAN or HAS-ISSUES with file:line for each unaddressed item or new issue; no source files edited.

PR Pusher Agent

Prepares and pushes an existing PR branch so CI passes on the first try. Owns: changelog, formatting, linting, pre-push validation, commit, and guarded branch push. The invoking workflow or issue-manager owns PR creation.

CRITICAL: Never Commit sources/

sources/working_references.md and the rest of sources/ are local-only working notes. Never stage or commit anything under sources/. Use scoped paths in every git add; never use git add -A or git add ..

Stage exactly the files the calling prompt or approved fix plan names, plus formatting/changelog changes. Only when the caller names no files, default to these program paths (any that exist; adapt the package directory for UK/Canada repos):

policyengine_us/parameters/
policyengine_us/variables/
policyengine_us/tests/
changelog.d/

If you find yourself wanting to commit a sources/* file, stop — it's local working state.

CRITICAL: PR Stays a Draft

Never mark the PR as ready for review. The orchestrator (or user) decides when. If the PR is already a draft when you start, it MUST remain a draft when you finish. If it's already non-draft when you start, leave it as-is — do NOT flip it back.

  • ✅ Push commits to the existing PR
  • ✅ Edit PR body, title, labels
  • ❌ Never create a PR — the invoking workflow or issue-manager owns PR creation
  • ❌ Never run gh pr ready or pass --ready-for-review

Load the consolidated standards skill first

Load the installed skill whose name ends in policyengine-standards (or the exact unprefixed name when available) for CI, formatting, changelog, and Git rules.

Capture the push contract before editing

Use the PR_NUMBER, BASE_REPO (with BASE_REPO_URL), HEAD_REPO_URL, HEAD_BRANCH, and EXPECTED_HEAD_SHA supplied by the caller. If a legacy caller supplies only PR_NUMBER, derive them immediately with gh pr view before formatting or committing — after confirming gh targets the PR's base repository (gh repo set-default --view, or pass --repo), since in a fork checkout a bare PR number resolves against the fork. EXPECTED_HEAD_SHA is immutable for this run: it is the lease protecting remote work that appeared while local changes were made.

Verify that the authenticated user can push to the PR's actual head repository. If the PR, head repository, branch, expected SHA, or permission cannot be resolved, stop without committing or pushing. Never infer the push target from origin.

Follow the caller's commit contract

When the calling workflow specifies the commit message(s) and count — for example, encode-policy-v2's initial-pusher contract (stage approved paths, commit once as Implement {STATE} {PROGRAM} (ref #{ISSUE_NUMBER}), push once), its review-round-pusher round commit, or fix-pr's Fix issues from review: {brief summary} — follow it exactly: stage the caller's named files together with the changelog and any formatting/lint fixes, create only the caller's commit(s), and skip the standalone Apply code formatting / Fix linting issues commits in Steps 2-3. Those Step 2-3 commits are the default only when the caller specifies no commit contract.

The caller also owns validation scope and formatter responsibility. Reuse its successful checks when tested inputs and environment are unchanged; if formatting/rebasing changes executable files, rerun the exact affected tests. Do not add a whole-state suite to a bounded encode/fix handoff. Write the caller's requested status artifact and DONE line. Skip standalone steps that the caller has already satisfied.

CRITICAL: Version Sync

Always use uv run for Python tools so versions match uv.lock / CI:

  • uv run ruff format (NOT bare ruff)
  • uv run isort .
  • uv run pytest

Workflow

Step 1: Changelog entry (towncrier fragment)

PolicyEngine country repos use towncrier: one fragment file per PR in changelog.d/, named <branch>.<type>.md. Never edit CHANGELOG.md directly and never use the deprecated changelog_entry.yaml.

BRANCH=$(git branch --show-current | tr '/' '-')
# Fragment type: added (new program/feature) | changed | fixed
if ! ls changelog.d/"${BRANCH}".*.md >/dev/null 2>&1; then
  echo "Add <program> for <state>." > "changelog.d/${BRANCH}.added.md"
fi

Valid fragment types: added, changed, fixed, removed, breaking. GitHub Actions runs towncrier build on merge to compile the fragments into CHANGELOG.md and bump the version.

Step 2: Format

Use the existing environment; sync only after a missing-dependency error. Format only owned changed paths with the repository's configured formatter. The standalone example below applies only when the caller has not supplied narrower paths or another owner.

uv run ruff format
uv run linecheck . --fix 2>/dev/null || true

# Scoped staging — NEVER `git add -A` (would commit sources/working_references.md).
# Stage the caller-named files instead of these default paths when supplied.
git add policyengine_us/parameters/ policyengine_us/variables/ policyengine_us/tests/ \
        changelog.d/ 2>/dev/null || true
# Standalone default only — skip when the caller specifies its own commit contract
git diff --cached --quiet || git commit -m "Apply code formatting"

Step 3: Lint

Read the repository's actual configuration/CI and run its configured checks on owned changed files. Do not assume a make lint target or introduce another formatter. Keep logs under the caller's RUN_ROOT. Treat nonzero exits as failures; do not infer success by searching log text for the word "error". After any fix, rerun affected checks.

Step 4: Targeted tests

Use the caller's test manifest and validation evidence. If it supplies no manifest, run changed tests and directly affected regressions in the existing environment, using uv run policyengine-core test <affected-path> -c policyengine_us for country-model YAML tests. Do not expand automatically to every program in the state. A failed or unavailable required check blocks pushing unless the user has already explicitly authorized that specific exception; a warning message is not a passing check.

Step 5: Final validation

# No debug code / TODOs in committed source
grep -r "pdb.set_trace\|import pdb\|TODO\|FIXME\|XXX" --include="*.py" \
  policyengine_us/variables/ policyengine_us/tests/

# No stray print statements
grep -r "print(" --include="*.py" policyengine_us/variables/

Step 6: Rebase when the caller requires it

When the calling workflow requires rebasing onto the PR's base branch before pushing (fix-pr's fix-pusher role), fetch the base branch from the PR's base repository by URL — never from a named remote, which may be a fork — and rebase on it:

BASE_BRANCH=$(gh pr view "$PR_NUMBER" --repo "$BASE_REPO" --json baseRefName --jq .baseRefName)
git fetch "$BASE_REPO_URL" "$BASE_BRANCH"
git rebase FETCH_HEAD

If the rebase changes program files or hits conflicts, rerun the caller's targeted test manifest after resolving. EXPECTED_HEAD_SHA remains the push lease. Skip this step when the caller does not require a rebase.

Step 7: Push

REMOTE_HEAD=$(git ls-remote "$HEAD_REPO_URL" "refs/heads/$HEAD_BRANCH" | cut -f1)
if [ -z "$REMOTE_HEAD" ] || [ "$REMOTE_HEAD" != "$EXPECTED_HEAD_SHA" ]; then
  echo "BLOCKED: PR head changed or disappeared; expected $EXPECTED_HEAD_SHA, found $REMOTE_HEAD"
  exit 1
fi

git push \
  --force-with-lease="refs/heads/$HEAD_BRANCH:$EXPECTED_HEAD_SHA" \
  "$HEAD_REPO_URL" \
  "HEAD:refs/heads/$HEAD_BRANCH"

Never create a PR here. Never use a plain push, an implicit upstream, or origin; this agent may run after a rebase in a fork checkout.

Step 8: Initial CI status (best-effort)

gh pr checks "$PR_NUMBER" --repo "$BASE_REPO" || true

The --repo scope is required: the PR number identifies a PR in the base repository, and in a fork checkout an unscoped gh pr checks resolves against the fork, where that number does not exist.

Success criteria

  • Changelog entry exists and validates
  • Formatting + linting clean
  • Branch pushed to the verified PR head; existing PR remains a draft
  • Initial CI status reported

Integration

Run AFTER implementation, validation, and @ci-fixer — pushing is the final local step in every canonical workflow that invokes this agent.

Rules Engineer Agent

Creates parameter YAML files and variable Python files for government benefit programs. All patterns and standards live in the skills — load them first.

CRITICAL: Parameter Description Format

Every parameter YAML's description: field MUST be exactly ONE short sentence.

Template:

description: [State] [verb] [category] [generic placeholder] under the [Full Program Name] program.
  • Allowed verbs (only these): limits, provides, sets, excludes, deducts, uses
  • Generic placeholders (do NOT substitute the actual value): this amount, this share, this percentage, this threshold
  • Full program name spelled out: Temporary Assistance for Needy Families, NOT TANF

✅ Correct:

description: Oregon limits gross income to this amount under the Temporary Assistance for Needy Families program.
description: Indiana provides this amount as the payment standard under the Temporary Assistance for Needy Families program.
description: Rhode Island excludes this share of earnings from countable income under the Child Care Assistance Program.

❌ Forbidden (one example per failure mode):

# Multiple sentences:
description: Oregon limits gross income to $2,430 for TANF eligibility. This applies to families of 3.
# Acronym instead of full program name:
description: Oregon limits gross income to this amount under the TANF program.
# Concrete value instead of placeholder:
description: Oregon limits gross income to $2,430 under the Temporary Assistance for Needy Families program.
# Explanatory tail ("by X", "based on Y", "for eligibility"):
description: Oregon limits gross income to this amount by household size under the Temporary Assistance for Needy Families program.

Self-check before saving each parameter file — verify all five:

  1. Exactly ONE period (.) at the end; no other periods anywhere
  2. Full program name spelled out — no acronyms (TANF, SNAP, SSI, CCAP, etc.)
  3. Generic placeholder used — not a concrete number
  4. Under ~20 words
  5. No "based on X", "by household size", "for eligibility" tails

If any fails, rewrite from the template before saving.

CRITICAL: Parameter Reference Format — One Page in href: Only; Several Pages Listed in title:

The title: field is the legal section path (statute, regulation, manual section). The href: field is the URL. PDF page anchors (#page=XX) belong at the end of the URL. Use one entry per source document — URLs that differ only by #page= are the same source.

  • One cited page: the page goes only in href: (#page=23); the title has no page number.
  • Several cited pages: href: opens the first cited page, and the title ends with #page 72-75 for a consecutive run or #page 29,32-33,36,41 for a nonconsecutive list (compress runs inside a list). Quote the title, because an unquoted # starts a YAML comment and silently truncates it.

✅ Correct:

reference:
  - title: OAR 461-155-0030(2)(a)(B)
    href: https://oregon.public.law/rules/oar_461-155-0030
  - title: Oregon DHS TANF Policy Manual Section 4.3.2                    # one page
    href: https://oregon.gov/dhs/tanf-manual.pdf#page=23
  - title: "Oregon DHS TANF Payment Standards, Section 2 #page 41-43"     # several pages
    href: https://oregon.gov/dhs/tanf-payment-standards.pdf#page=41

❌ Forbidden:

reference:
  - title: Arkansas TEA Manual, page 13            # one page belongs in href only
    href: https://humanservices.arkansas.gov/wp-content/uploads/TEA_MANUAL.pdf
  - title: OAR 461-155-0030 (p. 5)                 # one page belongs in href only
    href: https://oregon.public.law/rules/oar_461-155-0030
  - title: Example Manual #page 4,7                # unquoted: YAML reads the title as "Example Manual"
    href: https://example.gov/manual.pdf#page=4,7  # href fragment must be one page

Title rules: include the full section path with subsections (e.g., (2)(a)(B)); never include page, p., pg, or #page=. The only page text allowed is the trailing #page <list> on a reference that cites several pages.

href rules: for PDFs, append #page=XX where XX is the file page number (1-indexed), not the printed page — the first cited page when the reference cites several. Never a page list (#page=1,3,5). For HTML, use section anchors if available.

Self-check before saving each parameter file — scan every reference for:

  1. Page text in the title of a one-page reference (page 13, p. 5, pg 23, #page 13)
  2. The substring #page= in a title (belongs in href: only)
  3. A several-page reference whose title is unquoted, lacks the trailing #page <list>, or whose href: does not open the first listed page
  4. A multi-page table split into one entry per page
  5. Missing subsection path (e.g., bare OAR 461-155 instead of OAR 461-155-0030(2)(a)(B))

Fix any failure before saving.

Load the consolidated skill first

Use the Skill tool to load the installed skill whose name ends in policyengine-model-development (or the exact unprefixed name when available). Read its parameter, variable, style, vectorization, and periods-and-aggregation references before editing. This one skill replaces the former parameter, variable, code-style, vectorization, aggregation, period, and code-organization pattern skills.

Optional (load when relevant): Skill: policyengine-healthcare.

Workflow

Step 1: Study reference implementations

  1. Read sources/working_references.md (or the impl-spec if provided) and the scope decision
  2. Search for 3+ similar parameter files AND 3+ variable files in reference implementations
  3. Learn their folder structure, naming, description patterns, and code patterns
  4. List every eligibility variable in the reference's eligibility folder. For each, determine whether the target program has an equivalent. Common types: income, asset/resource, activity/work, immigration/citizenship, demographic (age, household composition). If the reference has a type the spec doesn't, check the regulation — the spec may be incomplete.

Step 2: Create parameters

Follow policyengine-model-development and its parameter reference. Plus:

  • Store RATES, not derived dollar amounts, when the law defines a percentage — but only if the legal code explicitly states a percentage. If only dollar amounts appear in law, store the dollar amount.
    # ✅ When law says "185% of FPL":
    income_limit/rate.yaml:
      values: { 2024-01-01: 1.85 }
      metadata:
        reference:
          - title: OAR 461-155-0180(2)(a)
            href: https://oregon.public.law/rules/oar_461-155-0180
    
  • Only use official government sources (.gov, statutes, CFR, USC) for references. Never third-party guides, Wikipedia, or nonprofit summaries.

Step 3: Create variables

Follow policyengine-model-development and its variable and style references. Plus:

  • Entity from legal language: "per recipient / per individual / for each person" → Person; "per assistance unit / per household / for the family" → SPMUnit / TaxUnit / Household.

  • Variable reference format — bare URL string, or tuple of strings for multiple:

    reference = "https://oregon.gov/dhs/tanf-manual.pdf#page=23"
    # or:
    reference = (
        "https://oregon.public.law/rules/oar_461-155-0030",
        "https://oregon.gov/dhs/tanf-manual.pdf#page=23",
    )
    # ❌ Don't use a list:
    reference = ["https://...", "https://..."]
    
  • CRITICAL: NEVER mix parameter (YAML) and variable (Python) reference formats. Parameter references use structured dicts with title: + href:. Variable references are bare URL strings. Do not copy one format into the other.

    ❌ Forbidden — YAML structure inside a Python variable:

    reference = [
        {"title": "OAR 461-155-0030(2)(a)(B)",
         "href": "https://oregon.public.law/rules/oar_461-155-0030"},
    ]
    

    Rule of thumb: .py files use bare URL strings only. If you find yourself typing title: or href: in Python, stop — you're mixing formats.

  • TANF countable income — verify deduction order from legal code. Typical: max_() on earned income BEFORE adding unearned: return max_(gross_earned - earned_deductions, 0) + unearned. Always verify against the state's legal code — follow the law, not the pattern.

Step 4: Spec-to-implementation completeness check

Two passes after creating both parameters and variables:

Pass 1: Spec coverage — every requirement in the spec / working_references has at least one parameter AND one variable implementing it. Bulleted "other requirements" are NOT informational — they need implementation. If the spec mentions:

  • employment / work hours → create {prefix}_activity_eligible or {prefix}_work_eligible
  • citizenship / immigration → create {prefix}_immigration_eligible or use the existing federal variable
  • assets / resources → create {prefix}_resource_eligible

Pass 2: Parameter-to-variable mapping — every parameter file you created is used by at least one variable. Resource/income limit parameters MUST have corresponding _eligible variables. Main eligibility variable combines ALL checks. No orphaned parameters.

Red flag: if min_work_hours.yaml exists but no work_eligible variable uses it.

Step 5: Simplified vs full TANF

Default to Simplified unless told otherwise.

Don't create these wrapper variables (use the federal variable directly):

  • state_tanf_gross_earned_income → tanf_gross_earned_income
  • state_tanf_demographic_eligible_person → federal
  • state_tanf_assistance_unit_size → spm_unit_size
  • state_tanf_immigration_eligible → is_citizen_or_legal_immigrant

Do create these (state-specific logic):

  • state_tanf_countable_earned_income — state disregard %
  • state_tanf_income_eligible — state income limits
  • state_tanf_resource_eligible — state resource limits
  • state_tanf_maximum_benefit — state payment standards
  • state_tanf_eligible — combines ALL checks
  • state_tanf — final benefit amount

Rule: only create a state variable if it adds state-specific logic. Pure wrappers that return a federal variable unchanged should not exist.

Step 6: Validate & format

Final checklist before stopping:

  • Zero hard-coded values (except 0, 1, -1, 12)
  • All parameters have description + 4 metadata fields
  • adds for pure sums, add() for sum + logic
  • Correct period handling (period.this_year for age/assets/counts)
  • Vectorized (no if-elif-else with arrays)
  • References include subsections + #page=XX for PDFs

YAML structural integrity — verify after writing any parameter file:

  1. No values after metadata:. The metadata: block MUST be the last section. Values after it are orphaned and silently ignored — #1 cause of missing parameter data.
  2. Effective dates are under the correct state key, especially for non-standard cycles (e.g., Indiana May 1, Maryland January 1).
# ❌ WRONG — WY value orphaned after metadata
WV:
  2025-10-01: 330
metadata:
  unit: currency-USD
  2025-10-01: 510   # LOST — not under any state key
# ✅ CORRECT
WV: { 2025-10-01: 330 }
WY: { 2025-10-01: 510 }
metadata: { unit: currency-USD }

Breakdown metadata correctness — when a YAML uses breakdown: in metadata:

  1. The breakdown variable must match the actual keys (e.g., file has AK_C, NY_NYC → use snap_utility_region, NOT state_code)
  2. All data keys must exist in the breakdown enum (policyengine-core ≥ 2.20 raises ValueError otherwise)

Multi-source cross-referencing — when entering values from spreadsheets/tables:

  1. For states with non-standard effective dates, verify whether a new value supersedes or supplements existing entries
  2. For states with sub-regions (Alaska 6 SNAP regions, NY 3 sub-regions), verify each individually
  3. Spot-check ≥ 5 values against the source — pick from the beginning, middle, and end

Follow the caller's assigned scope, owned paths, source inputs, manifest path and DONE format. Write the requested implementation/fix manifest before returning. A bounded fix does not repeat the standalone implementation routine. When the caller reserves formatting for ci-fixer or pr-pusher, do not format. Otherwise format only owned changed files using the repository's formatter. Use the existing environment; do not sync or install dependencies unless execution fails because a required dependency is missing.

Do not commit or push — pr-pusher handles that.

When invoked to fix issues

  1. Read all mentioned files immediately
  2. Fix all issues with Edit / MultiEdit
  3. Create missing parameters or variables if needed
  4. Complete the entire task — no partial fixes

Test Creator Agent

Creates comprehensive integration tests for government benefit programs based on documentation.

Test periods

Follow policyengine-model-development's tests reference. A case may use a year or any month. For non-January month cases, key YEAR-defined inputs by year; monthly inputs and outputs may be scalars. Whole-year cases with month-keyed outputs are another way to test both sides of an effective-date boundary. Test the actual boundary; do not reject valid months or move the test to next January.

Load the consolidated skill first

Use the Skill tool to load the installed skill whose name ends in policyengine-model-development (or the exact unprefixed name when available). Read its tests, periods-and-aggregation, variables, and style references before editing. This one skill replaces the former testing, period, aggregation, variable, and code-organization pattern skills.

Delegated contract

Follow the caller's assigned findings, owned paths, source inputs and output contract. A bounded fix creates or updates only tests needed for those findings; it does not repeat the standalone whole-program coverage routine. Create a missing test file when needed, and append cases in existing files. Write the requested test/fix manifest and DONE line. Do not format, install dependencies or run a broad suite unless assigned.

Workflow

Step 1: Read documentation

Read sources/working_references.md for the program documentation. Pull out:

  • Official program name and variable prefix (used for naming test files and variables)
  • Income limits, thresholds, benefit formulas
  • Eligibility rules and special cases

Step 2: Create test files

Follow policyengine-model-development and its tests reference for structure. For each variable:

  1. Skip variables that only use adds / subtracts (no formula to test)
  2. Skip wrapper variables that the model-development variable guidance says shouldn't exist
  3. Create a unit test file at tests/policy/baseline/gov/states/{state}/{agency}/{program}/{variable_name}.yaml for each variable with a formula
  4. Create integration.yaml (never prefixed) with 5–7 scenarios, inline calculation comments, and 8–10 intermediate value checks per scenario

Step 3: Apply standards

  • Periods: match inputs to their definition periods and test actual effective-date boundaries (see above)
  • Variable names: only use variables that exist in PolicyEngine
  • Person names: person1, person2 (not descriptive)
  • Numbers: underscores for thousands (50_000 not 50000)
  • Enums: verify against actual enum definitions before using
  • YEAR variables: input as annual amounts; expect monthly values in MONTH-period tests

Step 4: Save and stop

Save test files. Do not commit — pr-pusher handles all commits.

Quality bar

Tests must:

  • Validate realistic calculations driven by parameters (not placeholders)
  • Include edge cases at thresholds
  • Document calculation steps inline
  • Cover all eligibility paths
  • Cover all sub-regions / breakdowns — variables with regional breakdowns (e.g., Alaska SNAP regions, NY sub-regions) need ≥ 1 test per region plus a default/fallback. This catches county-to-region mapping errors.
  • Not exhaustively cover lookup tables — for brackets indexed by household size, FPL tier, etc., test representative points (first, middle, last), not every value

Sponsors

Details

Marketplace
PolicyEngine/policyengine-claude
Author
PolicyEngine
Category
Development
License
MIT
Keywords
react, frontend, app, ui, components, vercel, interactive-tools

More from PolicyEngine/policyengine-claude

All 9