PluginBench
MCP Server
Active
MIT

Iris MCP Server

io.github.iris-eval/mcp-server

RepositoryRegistry
How to choose

What this kind of tool is for

  • Hand your agent a structured way to call a model or an inference API.
  • Compare outputs from two different providers on the same prompt.
  • Wire retrieval, embeddings, or generation into an existing agent workflow.

How to compare servers like this

Provider coverage:
Match the tool's supported providers and models to the ones your project uses.
Cost exposure:
Model-backed tools spend real API budget; check how usage and limits are configured.
Maintenance signal:
The provider landscape shifts weekly; prefer tools updated within the last few months.
Config surface:
Prefer tools whose environment-variable or config keys are documented plainly.

This page carries your copy-paste config: pick your MCP client in the install section on this page and use the copy button.

What is the Iris MCP server?

Stop shipping agents on vibes. Score every agent output for quality, safety, and cost.

How to install Iris

Copy-paste configuration for popular MCP clients.

transport: stdio
Config generated by PluginBench — verify against the source before use.
Environment / auth
  • IRIS_API_KEY
    secret

    API key for HTTP authentication. Required to bind the HTTP transport or the dashboard beyond loopback (0.0.0.0, a LAN address, a container): without it the server refuses to start

  • IRIS_API_KEY_FILE

    Path to a file whose trimmed contents are the API key (the secret-file pattern Docker and Kubernetes mount). Set this or IRIS_API_KEY, not both; further keys and rotation are security.apiKeys in config.json

  • IRIS_ALLOW_UNAUTHENTICATED

    Set to 1 to run a non-loopback bind with NO API key on purpose (lifts the startup refusal; the network is then your boundary)

  • IRIS_DB_PATH

    SQLite database path

  • IRIS_SQLITE_DRIVER

    Which SQLite driver holds the database: native (better-sqlite3, the default) or node (Node's built-in node:sqlite, Node 22.13+). Unset: native, falling back to node when the native module cannot load

  • IRIS_LOG_LEVEL

    Log level: debug, info, warn, error

  • IRIS_HOME

    Directory for the database, custom rules and preferences (default ~/.iris)

  • IRIS_ANTHROPIC_API_KEY
    secret

    Enables evaluate_with_llm_judge and verify_citations with an Anthropic model. Optional: the deterministic rules never need a key

  • IRIS_OPENAI_API_KEY
    secret

    Enables evaluate_with_llm_judge and verify_citations with an OpenAI model. Optional: the deterministic rules never need a key

  • IRIS_LLM_JUDGE_MAX_COST_USD_PER_EVAL

    Hard cap on the worst-case cost of one judge or citation call (default 0.25)

  • IRIS_CITATION_ALLOW_FETCH

    Let verify_citations fetch URLs (off by default)

  • IRIS_DASHBOARD

    Start the dashboard alongside the server (same as --dashboard)

  • IRIS_DASHBOARD_PORT

    Dashboard port (default 6920); IRIS_PORT is the MCP HTTP transport port (default 3000)

  • IRIS_PORT

    MCP HTTP transport port (default 3000); only used with --transport http

  • IRIS_OTEL_ENDPOINT

    OTLP/HTTP collector URL; when set, every stored trace is also exported as OpenTelemetry spans (best-effort, asynchronous)

  • IRIS_CITATION_DOMAINS

    Comma-separated domain allowlist for verify_citations fetches (only consulted when IRIS_CITATION_ALLOW_FETCH is on)

  • IRIS_TRANSPORT

    Transport: stdio (default) or http; same as --transport

  • IRIS_HOST

    Bind host for the HTTP transport (default 127.0.0.1); only used with --transport http

  • IRIS_ALLOWED_ORIGINS

    Comma-separated Origin allowlist for the HTTP transport and dashboard CORS (default http://localhost:*)

  • IRIS_DASHBOARD_HOST

    Bind host for the dashboard (default 127.0.0.1); same as --dashboard-host

  • IRIS_OTEL_HEADERS

    Comma-separated key=value headers sent with every OTLP export (for example an auth token for your collector)

  • IRIS_OTEL_SERVICE_NAME

    service.name resource attribute on exported spans (default iris-eval)

  • IRIS_OTEL_TIMEOUT_MS

    Timeout in milliseconds for each OTLP export request

  • IRIS_WEBHOOK_URL

    The receiver of the webhook that fires on a moment (docs/webhooks.md); merged over notify.webhook in config.json

  • IRIS_WEBHOOK_SECRET
    secret

    The signing key of that webhook (any string, or whsec_ + base64); the iris format refuses to run without one

~/Library/Application Support/Claude/claude_desktop_config.json
JSON config
{
  "mcpServers": {
    "mcp-server": {
      "command": "npx",
      "args": [
        "-y",
        "@iris-eval/mcp-server"
      ],
      "env": {
        "IRIS_API_KEY": "<YOUR_IRIS_API_KEY>",
        "IRIS_API_KEY_FILE": "<YOUR_IRIS_API_KEY_FILE>",
        "IRIS_ALLOW_UNAUTHENTICATED": "<YOUR_IRIS_ALLOW_UNAUTHENTICATED>",
        "IRIS_DB_PATH": "<YOUR_IRIS_DB_PATH>",
        "IRIS_SQLITE_DRIVER": "<YOUR_IRIS_SQLITE_DRIVER>",
        "IRIS_LOG_LEVEL": "<YOUR_IRIS_LOG_LEVEL>",
        "IRIS_HOME": "<YOUR_IRIS_HOME>",
        "IRIS_ANTHROPIC_API_KEY": "<YOUR_IRIS_ANTHROPIC_API_KEY>",
        "IRIS_OPENAI_API_KEY": "<YOUR_IRIS_OPENAI_API_KEY>",
        "IRIS_LLM_JUDGE_MAX_COST_USD_PER_EVAL": "<YOUR_IRIS_LLM_JUDGE_MAX_COST_USD_PER_EVAL>",
        "IRIS_CITATION_ALLOW_FETCH": "<YOUR_IRIS_CITATION_ALLOW_FETCH>",
        "IRIS_DASHBOARD": "<YOUR_IRIS_DASHBOARD>",
        "IRIS_DASHBOARD_PORT": "<YOUR_IRIS_DASHBOARD_PORT>",
        "IRIS_PORT": "<YOUR_IRIS_PORT>",
        "IRIS_OTEL_ENDPOINT": "<YOUR_IRIS_OTEL_ENDPOINT>",
        "IRIS_CITATION_DOMAINS": "<YOUR_IRIS_CITATION_DOMAINS>",
        "IRIS_TRANSPORT": "<YOUR_IRIS_TRANSPORT>",
        "IRIS_HOST": "<YOUR_IRIS_HOST>",
        "IRIS_ALLOWED_ORIGINS": "<YOUR_IRIS_ALLOWED_ORIGINS>",
        "IRIS_DASHBOARD_HOST": "<YOUR_IRIS_DASHBOARD_HOST>",
        "IRIS_OTEL_HEADERS": "<YOUR_IRIS_OTEL_HEADERS>",
        "IRIS_OTEL_SERVICE_NAME": "<YOUR_IRIS_OTEL_SERVICE_NAME>",
        "IRIS_OTEL_TIMEOUT_MS": "<YOUR_IRIS_OTEL_TIMEOUT_MS>",
        "IRIS_WEBHOOK_URL": "<YOUR_IRIS_WEBHOOK_URL>",
        "IRIS_WEBHOOK_SECRET": "<YOUR_IRIS_WEBHOOK_SECRET>"
      }
    }
  }
}

Related MCP servers

Validate and convert JSONL fine-tuning data across 11 AI providers. 13 tools.

View repository →

MCP server for persistent, searchable memory for AI agents.

0
TypeScript
MIT
View repository →

Give brains to your small models. MCP server enforcing step-by-step Chain-of-Thought.

0
TypeScript
MIT
View repository →

MCP for Singularity Layer - discover listings, manage assets, run payment flows, and use agents

MEMemoryGuard logo

Local-first governed MCP memory backend for coding agents.

3
Python
MIT
View repository →

Threads tools for AI — generate viral posts, download videos, export profiles

View repository →