io.github.RudrenduPaul/agent-eval MCP Server
io.github.RudrenduPaul/agent-eval
What this kind of tool is for
- Hand your agent a structured way to call a model or an inference API.
- Compare outputs from two different providers on the same prompt.
- Wire retrieval, embeddings, or generation into an existing agent workflow.
How to compare servers like this
- Provider coverage:
- Match the tool's supported providers and models to the ones your project uses.
- Cost exposure:
- Model-backed tools spend real API budget; check how usage and limits are configured.
- Maintenance signal:
- The provider landscape shifts weekly; prefer tools updated within the last few months.
- Config surface:
- Prefer tools whose environment-variable or config keys are documented plainly.
This page carries your copy-paste config: pick your MCP client in the install section on this page and use the copy button.
What is the io.github.RudrenduPaul/agent-eval MCP server?
Statistical regression testing for LLM agents: p-value, effect size, and CI on behavior change.
How to install io.github.RudrenduPaul/agent-eval
Copy-paste configuration for popular MCP clients.
{
"mcpServers": {
"agent-eval": {
"command": "uvx",
"args": [
"agent-regress-cli",
"--from",
"agent-regress-cli",
"agent-regress-mcp"
]
}
}
}Related MCP servers
AI agent observability with deterministic record/replay for debugging agent failures.
Wraps the AgenticWorkspace CLI as a single generic MCP tool for repo workspace scans.
BYOK Reddit/YouTube research CLI with a tamper-evident audit log and an MCP server.
Signs, hash-chains, and verifies compute usage receipts via an MCP server, no issuer trust required.
Wraps the ContinuityGuard CLI as a single generic MCP tool for video-clip continuity scans.

Evaluates whether an AI agent is safe to operate internal web apps via an MCP run_suite tool.