retriEVAL MCP Server

io.github.hcarrillo001/retrieval-mcp

Repository Registry

What is the retriEVAL MCP server?

LLM evals as MCP tools: score outputs for faithfulness, relevancy, and hallucination.

How to install retriEVAL

Copy-paste configuration for popular MCP clients.

transport: stdio

Config generated by PluginBench — verify against the source before use.

Environment / auth

ANTHROPIC_API_KEY
secret
Judge model API key. Not needed if you set RETRIEVAL_JUDGE_BACKEND=ollama to run the judge locally.
RETRIEVAL_JUDGE_BACKEND
Which judge to score with: anthropic (default), openai, or ollama for a fully local judge.
RETRIEVAL_JUDGE_MODEL
Judge model id, e.g. claude-sonnet-4-6 or qwen2.5:14b for Ollama.
RETRIEVAL_BUDGET_USD
Spend cap for judge calls. Unset means no cap.

~/Library/Application Support/Claude/claude_desktop_config.json

{
  "mcpServers": {
    "retrieval-mcp": {
      "command": "docker",
      "args": [
        "run",
        "-i",
        "--rm",
        "ghcr.io/hcarrillo001/retrieval-mcp:1.0.0"
      ],
      "env": {
        "ANTHROPIC_API_KEY": "<YOUR_ANTHROPIC_API_KEY>",
        "RETRIEVAL_JUDGE_BACKEND": "<YOUR_RETRIEVAL_JUDGE_BACKEND>",
        "RETRIEVAL_JUDGE_MODEL": "<YOUR_RETRIEVAL_JUDGE_MODEL>",
        "RETRIEVAL_BUDGET_USD": "<YOUR_RETRIEVAL_BUDGET_USD>"
      }
    }
  }
}

Related MCP servers

Computer-Use Agents API

Active

Run and manage H Company's Computer-Use Agents from any MCP client with browser automation and custom tools.

Python

MIT

View repository →

io.github.hcpapi/housecall-pro-mcp

Active

Independent MCP server for the Housecall Pro API: jobs, estimates, invoices, price book, schedule.

TypeScript

MIT

View repository →

io.github.headlessherm-creator/polymath-megablaster-mcp

Active

Local deterministic calc MCP: exact math, dates, unit conversion, stats, subnet math.

JavaScript

MIT

View repository →

Apify data tools: Threads, Yelp, transcripts, Trends, Airbnb, Jumia

Active

Web data tools: Threads, Yelp, YouTube/TikTok transcripts, Google Trends, Airbnb, Jumia prices.

MIT

View repository →

io.github.healersfind-alt/organic-web-checker

Maintained

USDA NOP organic compliance checks — operation websites vs live OID certificate data.

View repository →

io.github.heavyblotto/caelus-mcp

Active

34-tool Caelus MCP for validated astrology: charts, transits, Vedic, facts, sky view, synthetic.

TypeScript

MIT

View repository →