PluginBench
MCP Server

io.github.jmshinhwa/robots-txt-ai-crawler-audit MCP Server

io.github.jmshinhwa/robots-txt-ai-crawler-audit

RepositoryRegistry
How to choose

What this kind of tool is for

  • Scan a repository or dependency tree for known vulnerabilities.
  • Audit secrets, permissions, or headers during a pre-release review.
  • Have your agent explain a suspicious script or config before you run it.

How to compare servers like this

Failure mode:
Vulnerability reports are probabilistic; read findings as leads to verify, not verdicts.
Data leaving the box:
Check whether files or code are uploaded anywhere before scanning anything private.
Maintenance signal:
Security rulesets age fast; stale tools miss new classes of issue.
Report clarity:
The best tools say what is wrong, how sure they are, and how to fix it — verify findings yourself.

This page carries your copy-paste config: pick your MCP client in the install section on this page and use the copy button.

What is the io.github.jmshinhwa/robots-txt-ai-crawler-audit MCP server?

Checks robots.txt line by line and says what each AI crawler token actually controls - training, AI

How to install io.github.jmshinhwa/robots-txt-ai-crawler-audit

Copy-paste configuration for popular MCP clients.

transport: stdio
Config generated by PluginBench — verify against the source before use.
~/Library/Application Support/Claude/claude_desktop_config.json
JSON config
{
  "mcpServers": {
    "robots-txt-ai-crawler-audit": {
      "command": "npx",
      "args": [
        "-y",
        "@readystack/robots-txt-ai-crawler-audit",
        "--mcp"
      ]
    }
  }
}

Related MCP servers

法定帳簿の保存期間を文書保存期間表で検査 — 労基法109条・安衛則51条・石綿則41条・雇用保険法施行規則143条・健保則34条(20 rules)

View repository →

Finds the end-of-life runtimes an AI assistant still writes for you — node:20, python3.9, ubuntu-22

View repository →

14 verificações ao XML SAF-T (PT) antes da entrega à AT: versão 1.04_01, ATCUD, dados do QR code, ha

View repository →

Reads security headers and CSP line by line in your config file and names the lines that silently do

View repository →

13 rules that decide whether a vulnerability report ever reaches you

View repository →

Find the lines your bank will refuse before you send the batch

View repository →