Llm Output Quality Monitor
A lightweight monitor for tracking LLM output quality, deployed on Cloudflare Workers and accessible via both REST and MCP protocols. The implementation uses streamable HTTP transport, making it straightforward to integrate into existing workflows without heavyweight dependencies. You'd reach for this when you need to programmatically assess or log the quality of responses from language models, whether you're building evaluation pipelines, debugging prompt engineering experiments, or maintaining quality metrics across production LLM integrations. The dual REST and MCP interface means you can query it from traditional HTTP clients or hook it directly into Claude's tool ecosystem.
- tools
- 5
- transport
- HTTP
- last updated
- May 16, 2026
Install
From registry metadata$ npx add-mcp https://api.lazy-mac.com/llm-output-quality-monitor/mcp --name llm-output-quality-monitorRun in your terminal. add-mcp asks which agents to add it to (Claude Code, Codex, Cursor and more). Add -g to install it for every project.
Review the command, arguments and environment values before installing. MCP servers run with your local permissions.
Other ways
Claude Code
$ claude mcp add --transport http llm-output-quality-monitor https://api.lazy-mac.com/llm-output-quality-monitor/mcpRun in your terminal. Add --scope user to make it available in every project.
Tools
Verified live · 5 toolsVerified live against the running server on Nov 6, 2026.
quality_validatorValidate LLM response quality based on length, format, and structure4 params
Validate LLM response quality based on length, format, and structure
maxLengthnumberminLengthnumberresponse*stringstrictFormatbooleandrift_detectorDetect quality drift between current and previous LLM responses3 params
Detect quality drift between current and previous LLM responses
currentResponse*stringpreviousResponse*stringthresholdnumberhallucination_scorerPattern-based heuristic risk scoring for LLM responses (0-100). Detects linguistic signals such as contradictory assertions, unsourced claims, and uncertainty markers. Not a semantic hallucination detector.2 params
Pattern-based heuristic risk scoring for LLM responses (0-100). Detects linguistic signals such as contradictory assertions, unsourced claims, and uncertainty markers. Not a semantic hallucination detector.
contextstringresponse*stringschema_enforcerValidate JSON response against schema2 params
Validate JSON response against schema
response*stringschema*objectconsistency_checkCheck consistency across multiple LLM responses1 params
Check consistency across multiple LLM responses
responses*arraySponsors
Agent CaffeineSponsoredClose your lid, come back to finished work
SnugSponsoredThe clipboard manager for the AI eraPromote your product hereReach 125k+ visitors monthly
WaxSponsoredBlock anything with supervision, without erasing your iPhonePromote your product hereReach 125k+ visitors monthly
knobsSponsoredSee your site on any phone, any day, in any languageReadme
llm-output-quality-monitor-api
Cloudflare Workers MCP server: llm-output-quality-monitor. REST + MCP (JSON-RPC 2.0). Live: api.lazy-mac.com/llm-output-quality-monitor
Details
- Repository
- github.com/lazymac2x/llm-output-quality-monitor-api
- Categories
- AI & LLM Tools, Cloud & Infrastructure, Monitoring & Observability
- Transport
- HTTP
- Registry
- MCP registry, active
- Tools verified
- Nov 6, 2026
More from lazymac2x
- Api Mock Server
- Interactive Api Playground
- Smart Data Extractor
- Mcpwatch
- A11y Scorer
- Agent Loop Detector
- Agent Memory
- Agent Trace Auditor
- Agent Workflow Engine
- Ai Agent Scratchpad
- Ai Budget Planner
- Ai Changelog Writer
- Ai Cost Optimizer
- Ai Eval
- Ai Guardrails
- Ai Model Router
- Ai Prompt Optimizer
- Ai Provider Status
- Ai Rate Limit Tracker
- Ai Token Counter
- Api Changelog Tracker
- Api Composition Gateway
- Api Contract Validator
- Api Diff Monitor
Related AI & LLM Tools MCP servers
View all
Agent Replaymdfifty50-boop/agent-replay-mcpRecord and replay AI agent execution for debugging0stars
Memoryshackleai/memory-mcpPersistent memory for AI coding tools. 11 tools for any MCP-compatible AI tool.0stars
MADB — Agent Memory Databasespshkar84/madb-meta-agents-dbDurable causal memory for AI agents — semantic recall, causal lineage, and skills. Local-first.0stars
RPCS-1 Agent Tuner & Translation Bridgetravisbergen2/rpcs1-sdkFind your AI agent's likely failure mode, get runtime settings, and clarify ambiguous prompts.0stars
Agent Bankwirternow/ai-agent-bank-mcp-serverFinancial infrastructure for AI agents: wallets, USDC transfers, lending, jobs on Polygon0stars
HIPAA Agentxbt786/hipaa-agent-aiHIPAA compliance AI agent — scan, grade, SRA, and generate compliance docs.0stars