Advanced Evaluation

muratcankoylan/agent-skills-for-context-engineering

If you're building LLM-as-judge systems, this gives you the production patterns that actually matter: position bias mitigation through double-pass evaluation, pairwise comparison versus direct scoring trade-offs, and rubric design that reduces variance. The taxonomy is clean (direct scoring for objective criteria, pairwise for subjective preference), and the bias landscape section names the gotchas you'll hit (length bias, self-enhancement, verbosity rewards). What I appreciate is the metric selection framework that matches statistical measures to task structure, plus the honest guidance on confidence calibration. It assumes you're past "can I use an LLM to evaluate outputs" and into "how do I make this reliable enough to ship."

skills.sh installs: 185
GitHub stars: 18.3K
last updated: May 26, 2026

View on GitHub

Install

$ npx -y skills add muratcankoylan/agent-skills-for-context-engineering --skill advanced-evaluation --agent claude-code

Installs into .claude/skills of the current project.

Files

Advanced Evaluation

This skill covers production-grade techniques for evaluating LLM outputs using LLMs as judges. It synthesizes research from academic papers, industry practices, and practical implementation experience into actionable patterns for building reliable evaluation systems.

Key insight: LLM-as-a-Judge is not a single technique but a family of approaches, each suited to different evaluation contexts. Choosing the right approach and mitigating known biases is the core competency this skill develops.