PluginBench
MCP Server
Active
Apache-2.0

retriEVAL MCP Server

io.github.hcarrillo001/retrieval-mcp

RepositoryRegistry
How to choose

What this kind of tool is for

  • Hand your agent a structured way to call a model or an inference API.
  • Compare outputs from two different providers on the same prompt.
  • Wire retrieval, embeddings, or generation into an existing agent workflow.

How to compare servers like this

Provider coverage:
Match the tool's supported providers and models to the ones your project uses.
Cost exposure:
Model-backed tools spend real API budget; check how usage and limits are configured.
Maintenance signal:
The provider landscape shifts weekly; prefer tools updated within the last few months.
Config surface:
Prefer tools whose environment-variable or config keys are documented plainly.

This page carries your copy-paste config: pick your MCP client in the install section on this page and use the copy button.

What is the retriEVAL MCP server?

LLM evals as MCP tools: score outputs for faithfulness, relevancy, and hallucination.

How to install retriEVAL

Copy-paste configuration for popular MCP clients.

transport: stdio
Config generated by PluginBench — verify against the source before use.
Environment / auth
  • ANTHROPIC_API_KEY
    secret

    Judge model API key. Not needed if you set RETRIEVAL_JUDGE_BACKEND=ollama to run the judge locally.

  • RETRIEVAL_JUDGE_BACKEND

    Which judge to score with: anthropic (default), openai, or ollama for a fully local judge.

  • RETRIEVAL_JUDGE_MODEL

    Judge model id, e.g. claude-sonnet-4-6 or qwen2.5:14b for Ollama.

  • RETRIEVAL_BUDGET_USD

    Spend cap for judge calls. Unset means no cap.

~/Library/Application Support/Claude/claude_desktop_config.json
JSON config
{
  "mcpServers": {
    "retrieval-mcp": {
      "command": "docker",
      "args": [
        "run",
        "-i",
        "--rm",
        "ghcr.io/hcarrillo001/retrieval-mcp:1.0.0"
      ],
      "env": {
        "ANTHROPIC_API_KEY": "<YOUR_ANTHROPIC_API_KEY>",
        "RETRIEVAL_JUDGE_BACKEND": "<YOUR_RETRIEVAL_JUDGE_BACKEND>",
        "RETRIEVAL_JUDGE_MODEL": "<YOUR_RETRIEVAL_JUDGE_MODEL>",
        "RETRIEVAL_BUDGET_USD": "<YOUR_RETRIEVAL_BUDGET_USD>"
      }
    }
  }
}

Related MCP servers

Run and manage H Company's Computer-Use Agents from any MCP client with browser automation and custom tools.

33
Python
MIT
View repository →

TokenLab MCP server: 31 default and 80 full-profile tools for models, media, tasks, and files.

0
JavaScript
MIT
View repository →

Persistent context for AI agents built fromlearned vectors with certainty scores.

2
View repository →

Behavioral memory layer that learns your preferences and context across Claude, Cursor, and Codex.

14
Python
MIT
View repository →

Independent MCP server for the Housecall Pro API: jobs, estimates, invoices, price book, schedule.

0
TypeScript
MIT
View repository →

Local deterministic calc MCP: exact math, dates, unit conversion, stats, subnet math.

0
JavaScript
MIT
View repository →