What is GPTBot?
GPTBot crawls public web pages that may be used to train OpenAI's generative AI foundation models. Disallowing it in robots.txt tells OpenAI your content shouldn't be used for training.
It does not decide whether ChatGPT can show your site in search answers. That is OAI-SearchBot, a separate user agent you control separately.
- Robots.txt token:
GPTBot - Purpose: Model training
- Feeds: training data for OpenAI's foundation models
Who runs GPTBot?
GPTBot is run by OpenAI. Everything on this page comes from OpenAI's own documentation: OpenAI: Overview of OpenAI crawlers. We last checked it on 21 October 2026.
OpenAI uses 2 other tokens, each with its own robots.txt group:
- OAI-SearchBot: AI search & answers (ChatGPT search answers)
- ChatGPT-User: User-requested fetch (pages ChatGPT opens when a user asks)
How to allow GPTBot in robots.txt
GPTBot is allowed unless a rule blocks it. If your * group blocks it, give it its own group:
User-agent: GPTBot
Allow: /
Disallow: /admin/A named group replaces your User-agent: * group for GPTBot, so copy in any private folders you still want blocked, like /admin/ above.
How to block GPTBot in robots.txt
User-agent: GPTBot
Disallow: /Tells OpenAI not to use your pages for model training from now on. ChatGPT search is unaffected.
To block one folder only, list just that path:
User-agent: GPTBot
Disallow: /blog/drafts/Things to know about GPTBot
- Blocking GPTBot while allowing OAI-SearchBot is the combination OpenAI itself gives as an example: out of training, still in ChatGPT search.
- If both GPTBot and OAI-SearchBot are allowed, OpenAI may reuse one crawl for both purposes.
- OpenAI publishes the IP ranges GPTBot crawls from, so you can tell real GPTBot visits from fakes.
- robots.txt only grants permission. A CDN or firewall can still block GPTBot even when your rules allow it.
Check every AI crawler at once
The checker above tests GPTBot and every other AI crawler on this list against your live robots.txt. The full AI Crawler Checker explains how to read the results. For help beyond robots.txt, see technical SEO, AI search optimisation and GEO services.
AI search & answers
- OAI-SearchBot (OpenAI): ChatGPT search answers
- Claude-SearchBot (Anthropic): Claude's search results
- PerplexityBot (Perplexity): Perplexity's search results and citations
- Googlebot (Google): Google Search, including AI Overviews and AI Mode
- Applebot (Apple): Siri, Spotlight and Safari search and answers
- Bingbot (Microsoft): Bing search and Microsoft Copilot answers
User-requested fetch
- ChatGPT-User (OpenAI): pages ChatGPT opens when a user asks
- Claude-User (Anthropic): pages Claude opens when a user asks
- Perplexity-User (Perplexity): pages Perplexity opens when a user asks
Model training
- ClaudeBot (Anthropic): training data for Anthropic's models
- Meta-ExternalAgent (Meta): training data for Meta's foundation AI models
- CCBot (Common Crawl): Common Crawl's open web archive, widely used as AI training data
- Amazonbot (Amazon): Amazon products and services, including Amazon AI models
Robots.txt control token
- Google-Extended (Google): Gemini model training and grounding
- Applebot-Extended (Apple): Apple foundation model training
FAQ
GPTBot questions
No. ChatGPT search uses OAI-SearchBot, and on-request visits use ChatGPT-User. Blocking GPTBot only opts your content out of OpenAI's model training.
robots.txt only controls what GPTBot fetches from now on. It can't change what was collected before you added the rule.
Yes. Use Disallow lines for the folders you want to keep out, for example Disallow: /blog/drafts/, and leave the rest allowed.
Check the visitor's IP address against the GPTBot IP ranges OpenAI publishes. Anyone can copy the GPTBot user agent string.
Done for you
Want us to fix it for you?
Send us the task. Senior marketers fix what the AI Crawler Checker found and make the rest of your site easy for Google and AI answers to understand.
