firecrawl-map
firecrawl/cli
Discover and list all URLs on a website with optional search filtering.
What is firecrawl-map?
Maps a website to discover its URL structure and pages. Use this when you need to find a specific page on a large site, see the site structure, or list all available URLs before scraping or crawling.
- List all URLs on a website up to a specified limit
- Filter URLs by search query to find specific pages
- Handle sitemaps with configurable strategies (include, skip, or only)
- Include or exclude subdomain URLs
- Output results as JSON or plain text to a file
How to install firecrawl-map
npx skills add null --skill firecrawl-mapHow to use firecrawl-map
- 1.Run `firecrawl map "<url>"` to get all URLs on a site
- 2.Add `--search "<query>"` to filter URLs by keyword
- 3.Use `--limit <n>` to cap the number of results returned
- 4.Add `--json` to output as JSON instead of plain text
- 5.Use `-o <path>` to save results to a file
- 6.Combine with firecrawl-scrape to scrape the discovered URLs
Use cases
- Find a specific documentation page on a large site before scraping it
- Get a complete list of all URLs on a domain to understand site structure
- Search for pages containing specific keywords (e.g., 'authentication') across a large site
- Discover all subpages before deciding which ones to crawl or scrape
- Map a site structure as the first step before bulk extraction
- Developers building web scrapers or crawlers
- Content researchers exploring large websites
- API documentation explorers
- Site structure analysts
firecrawl-map FAQ
Use `firecrawl map "<url>" --search "<keyword>"` to filter the site's URLs by your search term. For example, `--search "authentication"` will show only URLs containing that word.
Map discovers and lists all URLs on a site. Crawl extracts content from multiple pages in bulk. Map is useful for finding the right page first; crawl is better for extracting data from many pages at once.
Yes, add the `--include-subdomains` flag to include URLs from subdomains in your results.
Use the `-o <path>` option, for example: `firecrawl map "<url>" -o ./urls.json --json` to save as JSON or `firecrawl map "<url>" -o ./urls.txt` for plain text.
The common pattern is: use `map --search` to find the right URL, then use `firecrawl-scrape` to extract content from that URL.
Full instructions (SKILL.md)
Source of truth, from firecrawl/cli.
name: firecrawl-map description: | Discover and list all URLs on a website, with optional search filtering. Use this skill when the user wants to find a specific page on a large site, list all URLs, see the site structure, find where something is on a domain, or says "map the site", "find the URL for", "what pages are on", or "list all pages". Essential when the user knows which site but not which exact page. allowed-tools:
- Bash(firecrawl *)
- Bash(npx firecrawl *)
firecrawl map
Discover URLs on a site. Use --search to find a specific page within a large site.
When to use
- You need to find a specific subpage on a large site
- You want a list of all URLs on a site before scraping or crawling
- Step 3 in the workflow escalation pattern: search → scrape → map → crawl → interact
Quick start
# Find a specific page on a large site
firecrawl map "<url>" --search "authentication" -o .firecrawl/filtered.txt
# Get all URLs
firecrawl map "<url>" --limit 500 --json -o .firecrawl/urls.json
Options
| Option | Description |
|---|---|
--limit <n> | Max number of URLs to return |
--search <query> | Filter URLs by search query |
--sitemap <include|skip|only> | Sitemap handling strategy |
--include-subdomains | Include subdomain URLs |
--json | Output as JSON |
-o, --output <path> | Output file path |
Tips
- Map + scrape is a common pattern: use
map --searchto find the right URL, thenscrapeit. - Example:
map https://docs.example.com --search "auth"→ found/docs/api/authentication→scrapethat URL.
See also
- firecrawl-scrape — scrape the URLs you discover
- firecrawl-crawl — bulk extract instead of map + scrape
- firecrawl-download — download entire site (uses map internally)
Related skills
More from firecrawl/cli and the wider catalog.
firecrawl
Search, scrape, and interact with the web via Firecrawl CLI—real-time content extraction and monitoring.
firecrawl-scrape
Extract clean markdown from any URL, including JavaScript-rendered pages.
firecrawl-search
Web search with full page content extraction—find articles, research topics, and discover sources beyond snippets.
firecrawl-crawl
Bulk extract content from entire websites or site sections with depth and path filtering.
firecrawl-agent
AI-powered autonomous data extraction from complex websites, returning structured JSON.
firecrawl-download
Download entire websites as local markdown, screenshots, or multiple formats organized in directories.