PluginBench
MCP Server
Active
Apache-2.0

io.github.RudrenduPaul/agent-eval MCP Server

io.github.RudrenduPaul/agent-eval

RepositoryRegistry
How to choose

What this kind of tool is for

  • Hand your agent a structured way to call a model or an inference API.
  • Compare outputs from two different providers on the same prompt.
  • Wire retrieval, embeddings, or generation into an existing agent workflow.

How to compare servers like this

Provider coverage:
Match the tool's supported providers and models to the ones your project uses.
Cost exposure:
Model-backed tools spend real API budget; check how usage and limits are configured.
Maintenance signal:
The provider landscape shifts weekly; prefer tools updated within the last few months.
Config surface:
Prefer tools whose environment-variable or config keys are documented plainly.

This page carries your copy-paste config: pick your MCP client in the install section on this page and use the copy button.

What is the io.github.RudrenduPaul/agent-eval MCP server?

Statistical regression testing for LLM agents: p-value, effect size, and CI on behavior change.

How to install io.github.RudrenduPaul/agent-eval

Copy-paste configuration for popular MCP clients.

transport: stdio
Config generated by PluginBench — verify against the source before use.
~/Library/Application Support/Claude/claude_desktop_config.json
JSON config
{
  "mcpServers": {
    "agent-eval": {
      "command": "uvx",
      "args": [
        "agent-regress-cli",
        "--from",
        "agent-regress-cli",
        "agent-regress-mcp"
      ]
    }
  }
}

Related MCP servers

AI agent observability with deterministic record/replay for debugging agent failures.

0
Python
Apache-2.0
View repository →

Wraps the AgenticWorkspace CLI as a single generic MCP tool for repo workspace scans.

0
Python
Apache-2.0
View repository →

BYOK Reddit/YouTube research CLI with a tamper-evident audit log and an MCP server.

0
Python
Apache-2.0
View repository →

Signs, hash-chains, and verifies compute usage receipts via an MCP server, no issuer trust required.

0
Python
Apache-2.0
View repository →

Wraps the ContinuityGuard CLI as a single generic MCP tool for video-clip continuity scans.

0
Python
Apache-2.0
View repository →

Evaluates whether an AI agent is safe to operate internal web apps via an MCP run_suite tool.

1
Python
Apache-2.0
View repository →