What is PerplexityBot?
PerplexityBot crawls the web to surface and link websites in Perplexity's search results. Perplexity recommends allowing it in robots.txt.
Perplexity says PerplexityBot isn't used to crawl content for AI foundation models, so blocking it only reduces your visibility in Perplexity.
- Robots.txt token:
PerplexityBot - Purpose: AI search & answers
- Feeds: Perplexity's search results and citations
Who runs PerplexityBot?
PerplexityBot is run by Perplexity. Everything on this page comes from Perplexity's own documentation: Perplexity: Perplexity crawlers. We last checked it on 21 October 2026.
Perplexity uses one other token, each with its own robots.txt group:
- Perplexity-User: User-requested fetch (pages Perplexity opens when a user asks)
How to allow PerplexityBot in robots.txt
PerplexityBot is allowed unless a rule blocks it. If your * group blocks it, give it its own group:
User-agent: PerplexityBot
Allow: /
Disallow: /admin/A named group replaces your User-agent: * group for PerplexityBot, so copy in any private folders you still want blocked, like /admin/ above.
How to block PerplexityBot in robots.txt
User-agent: PerplexityBot
Disallow: /Your pages stop being surfaced and linked in Perplexity's search results.
To block one folder only, list just that path:
User-agent: PerplexityBot
Disallow: /staging/Things to know about PerplexityBot
- Perplexity says robots.txt changes can take up to 24 hours to reach its systems.
- Each Perplexity robots.txt setting works independently: blocking PerplexityBot doesn't block Perplexity-User.
- Perplexity publishes PerplexityBot's IP ranges as JSON and gives setup steps for allowlisting it in Cloudflare and AWS WAF.
- robots.txt only grants permission. A CDN or firewall can still block PerplexityBot even when your rules allow it.
Check every AI crawler at once
The checker above tests PerplexityBot and every other AI crawler on this list against your live robots.txt. The full AI Crawler Checker explains how to read the results. For help beyond robots.txt, see technical SEO, AI search optimisation and GEO services.
AI search & answers
- OAI-SearchBot (OpenAI): ChatGPT search answers
- Claude-SearchBot (Anthropic): Claude's search results
- Googlebot (Google): Google Search, including AI Overviews and AI Mode
- Applebot (Apple): Siri, Spotlight and Safari search and answers
- Bingbot (Microsoft): Bing search and Microsoft Copilot answers
User-requested fetch
- ChatGPT-User (OpenAI): pages ChatGPT opens when a user asks
- Claude-User (Anthropic): pages Claude opens when a user asks
- Perplexity-User (Perplexity): pages Perplexity opens when a user asks
Model training
- GPTBot (OpenAI): training data for OpenAI's foundation models
- ClaudeBot (Anthropic): training data for Anthropic's models
- Meta-ExternalAgent (Meta): training data for Meta's foundation AI models
- CCBot (Common Crawl): Common Crawl's open web archive, widely used as AI training data
- Amazonbot (Amazon): Amazon products and services, including Amazon AI models
Robots.txt control token
- Google-Extended (Google): Gemini model training and grounding
- Applebot-Extended (Apple): Apple foundation model training
FAQ
PerplexityBot questions
No. Perplexity says PerplexityBot is not used to crawl content for AI foundation models.
Perplexity says changes can take up to 24 hours to be reflected.
Possibly. Perplexity-User fetches pages when a user asks a question, and Perplexity says it generally ignores robots.txt.
If you want to be cited in Perplexity, yes. Perplexity recommends allowing it, and it isn't used for model training.
Done for you
Want us to fix it for you?
Send us the task. Senior marketers fix what the AI Crawler Checker found and make the rest of your site easy for Google and AI answers to understand.
