What this kind of tool is for
- Let your agent search the web or a document set and cite what it found.
- Pull structured facts out of pages instead of copy-pasting by hand.
- Track changes on a topic over days without re-reading everything.
How to compare servers like this
- Source coverage:
- Check which engines, sites, or document types the tool can actually reach.
- Citation behavior:
- Prefer tools that return links you can open, not just summarized claims.
- Terms of service:
- Some sources forbid automated access; make sure the tool stays inside them.
- Maintenance signal:
- Scrapers break silently when pages change; recent commits are the best proxy.
This page carries your copy-paste config: pick your MCP client in the install section on this page and use the copy button.
What is the TheCrawler MCP server?
Universal web scraper with LLM-ready markdown, RAG chunking, PDF/DOCX support.
How to install TheCrawler
Copy-paste configuration for popular MCP clients.
{
"mcpServers": {
"thecrawler": {
"command": "npx",
"args": [
"-y",
"thecrawler"
]
}
}
}Related MCP servers

ReturnCheck
Can this product actually be returned? The merchant's exact clause, its URL and the date.
AI brand monitor: what ChatGPT and Claude say about your brand, press tracking, executive briefs.
View repository →
NHC MCP Server
NHC storm tracks, forecast advisories, watches/warnings, and HURDAT2 data
NWS surface wind observations (ASOS/AWOS) via Weather.gov and Iowa Environmental Mesonet APIs

OpenLMNP
Accounting MCP for French LMNP furnished rentals: income, expenses, depreciation, tax return & FEC
Create, edit, review, and explicitly publish Live or Snapshot Markdown Documents in mdedit.ai.
View repository →
