Llm Evaluation

LLM Evaluation

Master comprehensive evaluation strategies for LLM applications, from automated metrics to human evaluation and A/B testing.

Do not use this skill when

• The task is unrelated to llm evaluation
• You need a different domain or tool outside this scope

Instructions

• Clarify goals, constraints, and required inputs.
• Apply relevant best practices and validate outcomes.
• Provide actionable steps and verification.
• Use the examples below; no separate implementation playbook is bundled.

Use this skill when

• Measuring LLM application performance systematically
• Comparing different models or prompts
• Detecting performance regressions before deployment
• Validating improvements from prompt changes
• Building confidence in production systems
• Establishing baselines and tracking progress over time
• Debugging unexpected model behavior

Core Evaluation Types

1. Automated Metrics

Fast, repeatable, scalable evaluation using computed scores.