Agent Crawl Readiness MCP Server

io.github.sadri-dridi/agent-crawl-readiness

Repository Registry

What is the Agent Crawl Readiness MCP server?

Bounded crawl-readiness check for agent access: robots.txt, sitemap.xml, llms.txt, homepage.

How to install Agent Crawl Readiness

Copy-paste configuration for popular MCP clients.

transport: http

Config generated by PluginBench — verify against the source before use.

~/.cursor/mcp.json

{
  "mcpServers": {
    "agent-crawl-readiness": {
      "url": "https://agent-observatory-sensor.nolimit-observatory.workers.dev/s/agent-crawl-readiness/mcp"
    }
  }
}

Related MCP servers

Agent Discovery Index

Active

Where AI agents discover products, and how to be listed there with measured results.

View repository →

Agent Discovery Placement

Active

Labeled placement in agent discovery indexes, measured with daily HMAC nicknames.

View repository →

Agent Facing Catalog

Active

Machine catalog for products that want agent clients: surfaces, hops, discovery paths.

View repository →

Agent First Listing

Active

Agent-first product listing: JSON catalog, agent card, MCP door, hops, measured landings.

View repository →

Agent Hops For Your Site

Active

Hops are 302s an agent followed to your site. Count them, attribute them, grow them.

View repository →

Agent Lead Generation

Active

Leads from agents: hops attributed with a per-hop utm_id from discovery to trial to invoice.

View repository →