io.github.cyanheads/evals-mcp-server MCP Server

io.github.cyanheads/evals-mcp-server

Repository Registry

What is the io.github.cyanheads/evals-mcp-server MCP server?

Author verifiable eval records through a draft→review→revise→submit loop with enforced graders.

How to install io.github.cyanheads/evals-mcp-server

Copy-paste configuration for popular MCP clients.

transport: stdio

Config generated by PluginBench — verify against the source before use.

Environment / auth

EVALS_DATA_DIR
Root folder for record JSON. The store manages drafts/, submitted/, and exports/ subdirs under it.
EVALS_REQUIRE_CONFIRMATION
When 'true', evals_submit_draft requests human confirmation before finalizing.
EVALS_DEFAULT_LICENSE
Default metadata.license applied when a draft omits one.
EVALS_CAPTURE_DIR
Directory of framework-written tool-call captures; when set, captures EvalsIDs resolve to full dumps.
MCP_LOG_LEVEL
Sets the minimum log level for output (e.g., 'debug', 'info', 'warn').
MCP_HTTP_HOST
The hostname for the HTTP server.
MCP_HTTP_PORT
The port to run the HTTP server on.
MCP_HTTP_ENDPOINT_PATH
The endpoint path for the MCP server.
MCP_AUTH_MODE
Authentication mode to use: 'none', 'jwt', or 'oauth'.

~/Library/Application Support/Claude/claude_desktop_config.json

{
  "mcpServers": {
    "evals-mcp-server": {
      "command": "node",
      "args": [
        "-y",
        "@cyanheads/evals-mcp-server",
        "run",
        "start:stdio"
      ],
      "env": {
        "EVALS_DATA_DIR": "<YOUR_EVALS_DATA_DIR>",
        "EVALS_REQUIRE_CONFIRMATION": "<YOUR_EVALS_REQUIRE_CONFIRMATION>",
        "EVALS_DEFAULT_LICENSE": "<YOUR_EVALS_DEFAULT_LICENSE>",
        "EVALS_CAPTURE_DIR": "<YOUR_EVALS_CAPTURE_DIR>",
        "MCP_LOG_LEVEL": "<YOUR_MCP_LOG_LEVEL>",
        "MCP_HTTP_HOST": "<YOUR_MCP_HTTP_HOST>",
        "MCP_HTTP_PORT": "<YOUR_MCP_HTTP_PORT>",
        "MCP_HTTP_ENDPOINT_PATH": "<YOUR_MCP_HTTP_ENDPOINT_PATH>",
        "MCP_AUTH_MODE": "<YOUR_MCP_AUTH_MODE>"
      }
    }
  }
}

Related MCP servers

io.github.cyanheads/exchange-rates-mcp-server

Active

Convert currencies, get FX rates, and query historical ECB exchange rate data.

TypeScript

Apache-2.0

View repository →

io.github.cyanheads/faa-aircraft-registry-mcp-server

Active

Offline, keyless lookup of the US civil aircraft registry — decode N-numbers, search records.

TypeScript

Apache-2.0

View repository →

io.github.cyanheads/faostat-mcp-server

Active

UN FAOSTAT global food & agriculture statistics over a local SQLite mirror, via MCP.

TypeScript

Apache-2.0

View repository →

io.github.cyanheads/fbi-crime-mcp-server

Active

FBI Crime Data Explorer — UCR estimates, NIBRS breakdowns, hate crimes, arrests, and agency data.

TypeScript

View repository →

io.github.cyanheads/fcc-broadband-mcp-server

Active

FCC broadband availability, coverage analysis, and digital divide data for US geographies.

TypeScript

View repository →

io.github.cyanheads/fdic-banks-mcp-server

Active

Search FDIC-insured banks, Call Report financials, peer rankings, failures, deposit market share.

TypeScript

Apache-2.0

View repository →