Conkurrence
Adds consensus measurement tools to Claude using Fleiss' kappa and bootstrap confidence intervals to check if AI models agree with themselves or each other. The eight MCP tools let you run multi-model evaluations across Bedrock, OpenAI, and Gemini, generate statistical reports, compare runs over time, and estimate costs before executing. The self-consistency mode is handy because it uses MCP Sampling to test the host model without external API keys. You'd reach for this when you need statistically rigorous validation that an AI is giving consistent answers, especially for high-stakes applications where agreement matters more than speed. Includes schema validation and AI-powered schema suggestion from your data.
- tools
- 7
- transport
- STDIO
- last updated
- Oct 3, 2026
Tools
7 toolsPublic tool metadata for what this MCP can expose to an agent.
conkurrence_runExecute an evaluation across multiple AI raters
Execute an evaluation across multiple AI raters
No parameter schema in public metadata yet.
conkurrence_reportGenerate a detailed markdown report
Generate a detailed markdown report
No parameter schema in public metadata yet.
conkurrence_compareSide-by-side comparison of two runs
Side-by-side comparison of two runs
No parameter schema in public metadata yet.
conkurrence_trendTrack agreement over multiple runs
Track agreement over multiple runs
No parameter schema in public metadata yet.
conkurrence_suggestAI-powered schema suggestion from your data
AI-powered schema suggestion from your data
No parameter schema in public metadata yet.
conkurrence_validate_schemaValidate a schema before running
Validate a schema before running
No parameter schema in public metadata yet.
conkurrence_estimateEstimate cost and token usage
Estimate cost and token usage
No parameter schema in public metadata yet.
Sponsors
Agent CaffeineSponsoredClose your lid, come back to finished work
SnugSponsoredThe clipboard manager for the AI eraPromote your product hereReach 125k+ visitors monthly
WaxSponsoredBlock anything with supervision, without erasing your iPhonePromote your product hereReach 125k+ visitors monthly
knobsSponsoredSee your site on any phone, any day, in any languageReadme
ConKurrence
Multi-model rating for AI evaluation: measures agreement among model raters (Fleiss' κ, Kendall's W) to find contested items for expert review.
What agreement tells you: where your raters and your criteria are consistent, and where they are not — which items are contested, and whether disagreement comes from the criteria or the raters.
What it does not tell you: that the raters are right. Agreement is reliability, not validity. Models can agree confidently on a wrong answer. Use ConKurrence to decide where an expert's judgment is needed and to diagnose your evaluation criteria — not to replace expert-labelled ground truth.
Status: maintenance mode. ConKurrence is developed by JE Vectors LLC as the evaluation instrument behind Inqura. It is not accepting purchases and carries no support commitment.
Install
npm install -g conkurrence
MCP Server
Use ConKurrence as an MCP server in Claude Desktop or any MCP-compatible client:
npx conkurrence mcp
Claude Desktop Configuration
Add to your claude_desktop_config.json:
Details
- Repository
- github.com/alligatorc0der/conkurrence
- Package
conkurrence- Transport
- STDIO
- Registry
- MCP registry, active