Phoenix Evals
If you're building LLM applications, you need evals, and Phoenix gives you a practical framework for both. The approach is sensible: start with error analysis to understand what's actually failing, build code-based evaluators for deterministic checks first, then layer in LLM judges for nuanced cases. The skill covers pre-built evaluators for common patterns like RAG, but the real value is in helping you build custom ones from your actual failures. One thing I appreciate: they're explicit about validating your evaluators against human labels (aiming for 80%+ accuracy) and prefer binary pass/fail over fuzzy scoring scales. Works in both Python and TypeScript, requires a Phoenix server running.
- skills.sh installs
- 1.7K
- GitHub stars
- 12.2K
- last updated
- Aug 21, 2026
Install
$ npx -y skills add arize-ai/phoenix --skill phoenix-evals --agent claude-codeInstalls into .claude/skills of the current project.
Files
If you're building LLM applications, you need evals, and Phoenix gives you a practical framework for both. The approach is sensible: start with error analysis to understand what's actually failing, build code-based evaluators for deterministic checks first, then layer in LLM judges for nuanced cases. The skill covers pre-built evaluators for common patterns like RAG, but the real value is in helping you build custom ones from your actual failures. One thing I appreciate: they're explicit about validating your evaluators against human labels (aiming for 80%+ accuracy) and prefer binary pass/fail over fuzzy scoring scales. Works in both Python and TypeScript, requires a Phoenix server running.
Browse the repository on GitHubSponsors
Agent CaffeineSponsoredClose your lid, come back to finished work
SnugSponsoredThe clipboard manager for the AI eraPromote your product hereReach 125k+ visitors monthly
WaxSponsoredBlock anything with supervision, without erasing your iPhonePromote your product hereReach 125k+ visitors monthly
knobsSponsoredSee your site on any phone, any day, in any languageDetails
- Repository
- github.com/arize-ai/phoenix
- Categories
- DevOps & CI/CD, AI & Agent Building
- First seen
- Jun 3, 2026
More from arize-ai/phoenix
All 3Recommended
More DevOps & CI/CD
ui-designerdaymade/claude-code-skillsExtract design systems from reference UI images and generate implementation-ready UI design prompts. Use when users provide UI screenshots/mockups and want to c1.7Kskills.sh installs
design-engineeringrefoundai/lenny-skillsHelp users understand and build design engineering capabilities. Use when someone is creating a design engineering function, hiring design engineers, or bridgin1.7Kskills.sh installs
write-a-skillvinvcn/mattpocock-skills-zh-cn创建结构正确、支持 progressive disclosure 并带 bundled resources 的新 agent skills。Use when user wants to create, write, or build a new skill.1.7Kskills.sh installsproduct-operationsrefoundai/lenny-skillsHelp users build and scale product operations functions. Use when someone is scaling a product team, struggling with cross-functional coordination, needs to sta1.7Kskills.sh installs
aws-cdk-python-setupgithub/awesome-copilotSetup and initialization guide for developing AWS CDK (Cloud Development Kit) applications in Python. This skill enables users to configure environment prerequi1.7Kskills.sh installs
paper-compilationlingzhi227/agent-research-skillsCompile LaTeX papers to PDF with automatic error detection, chktex style checking, and citation/reference validation. Runs the full pdflatex + bibtex pipeline. 1.7Kskills.sh installs