Phoenix Evals
If you're building LLM apps and need to move beyond vibes-based testing, this gives you the Phoenix evaluation framework. It's refreshingly opinionated: error analysis before automation, code evaluators before LLM judges, binary pass/fail over fuzzy scoring. You get pre-built evaluators for common cases like RAG, tools to build custom ones in Python or TypeScript, and crucially, validation workflows to check your LLM judges against human labels (they recommend 80% true positive/negative rates minimum). The workflow guides are practical, from trace sampling through production guardrails. Worth noting it requires running a Phoenix server, so there's infrastructure to manage.
- skills.sh installs
- 1K
- GitHub stars
- 41K
- last updated
- Aug 21, 2026
Install
$ npx -y skills add github/awesome-copilot --skill phoenix-evals --agent claude-codeInstalls into .claude/skills of the current project.
Files
Phoenix Evals
Build evaluators for AI/LLM applications. Code first, LLM for nuance, validate against humans.
Quick Reference
Sponsors
Agent CaffeineSponsoredClose your lid, come back to finished work
SnugSponsoredThe clipboard manager for the AI eraPromote your product hereReach 125k+ visitors monthly
WaxSponsoredBlock anything with supervision, without erasing your iPhonePromote your product hereReach 125k+ visitors monthly
knobsSponsoredSee your site on any phone, any day, in any languageDetails
- Repository
- github.com/github/awesome-copilot
- Categories
- DevOps & CI/CD, Git & Pull Requests, AI & Agent Building
- First seen
- Apr 16, 2026
More from github/awesome-copilot
All 318- Phoenix Tracing1K
- Onboard Context Matic1K
- Phoenix Cli1K
- React19 Source Patterns993
- Python Pypi Package Builder950
- Salesforce Apex Quality885
- React Audit Grep Patterns884
- Salesforce Flow Design839
- Github Release790
- Flowstudio Power Automate Monitoring787
- Flowstudio Power Automate Governance779
- Salesforce Component Standards770
- React18 Dep Compatibility766
- React18 Batching Patterns756
- React18 Legacy Context747
- From The Other Side Vega746
- React18 Lifecycle Patterns745
- React18 String Refs743
- React18 Enzyme To Rtl742
- Code Tour739
- Linkedin Post Formatter361
- Ui Screenshots212
- Data Breach Blast Radius110
- Sql Server Table Reconciliation104
Recommended
More DevOps & CI/CD
phoenix-tracinggithub/awesome-copilotOpenInference semantic conventions and instrumentation for Phoenix AI observability. Use when implementing LLM tracing, creating custom spans, or deploying to p1Kskills.sh installs
nextjsjezweb/claude-skillsBuild Next.js 16 apps with App Router, Server Components/Actions, Cache Components ("use cache"), and async route params. Includes proxy.ts and React 19.2. Prev1Kskills.sh installs
prompt-guardorchestra-research/ai-research-skillsMeta's 86M prompt injection and jailbreak detector. Filters malicious prompts and third-party data for LLM apps. 99%+ TPR, <1% FPR. Fast (<2ms GPU). Multilingua1Kskills.sh installsdeepspeedorchestra-research/ai-research-skillsExpert guidance for distributed training with DeepSpeed - ZeRO optimization stages, pipeline parallelism, FP16/BF16/FP8, 1-bit Adam, sparse attention1Kskills.sh installsdspyorchestra-research/ai-research-skillsBuild complex AI systems with declarative programming, optimize prompts automatically, create modular RAG systems and agents with DSPy - Stanford NLP's framewor1Kskills.sh installsdesign-loopjezweb/claude-skillsAutonomous multi-page site builder using a baton-passing loop. Each iteration reads a task from .design/next-prompt.md, generates a page in HTML/Tailwind, integ1Kskills.sh installs