io.github.ryuamarines/robots-txt-checker MCP Server
io.github.ryuamarines/robots-txt-checker
What this kind of tool is for
- Automate a build, test, or deployment step you repeat by hand.
- Review or refactor code in an existing repository without leaving your agent.
- Bridge an internal service or CLI into the agent toolset.
How to compare servers like this
- Scope of control:
- Repos and CI are hard to undo; prefer tools that ask before anything irreversible.
- Environment fit:
- Match the runtime it expects (Node, Python, a container) to what you actually run.
- Maintenance signal:
- Check the last-commit date — dev tooling that lags upstream breaks first.
- Config surface:
- Prefer tools whose environment-variable or config keys are documented plainly.
This page carries your copy-paste config: pick your MCP client in the install section on this page and use the copy button.
What is the io.github.ryuamarines/robots-txt-checker MCP server?
Checks if a URL path is allowed by robots.txt. x402 payment required (testnet USDC).
How to install io.github.ryuamarines/robots-txt-checker
Copy-paste configuration for popular MCP clients.
{
"mcpServers": {
"robots-txt-checker": {
"url": "https://faceted-napped-debtless.ngrok-free.dev/robots-txt-checker/mcp"
}
}
}Related MCP servers
Converts text titles into URL-safe slugs and flags duplicates. x402 payment required (testnet USDC).
Normalizes and deduplicates a list of URLs. x402 payment required (testnet USDC, Base Sepolia).
Checks current App Store version by bundle ID. x402 payment required (testnet USDC).
Extracts emails from texts. x402 payment required (testnet USDC).

io.github.ryudi84/api
MCP server for generating REST API boilerplate and Express endpoints.

io.github.ryudi84/api-tester
MCP server for API testing: HTTP requests, cURL generation, health checks.