OAI-SearchBot vs GPTBot vs ChatGPT-User

OAI-SearchBot is for ChatGPT search discovery, GPTBot is for crawling content that may be used to improve OpenAI’s generative AI foundation models, and ChatGPT-User fetches pages for specific user actions. Their purposes and controls are independent.

Key takeaways

• Allow OAI-SearchBot if you want eligible pages available to ChatGPT search discovery.
• Control model-training crawling separately with GPTBot.
• Do not use ChatGPT-User as a substitute for the search crawler policy; OpenAI describes it as user-triggered and says robots.txt may not apply.
• An allowed crawler can still encounter HTTP errors, page-level directives, authentication or rendering problems.
• Access never guarantees indexing, retrieval, citation, referral traffic or conversion.

The three OpenAI agents serve different jobs

OpenAI says the automatic search and training settings are independent. It may use the result of one crawl for both allowed use cases to avoid duplicate requests, but the webmaster-facing policy choices remain separate.

For the normalized definitions Scopeva uses, see Crawler types.

Example: allow search, block training

For a task-focused deployment checklist, use How to block GPTBot while allowing OAI-SearchBot.

User-agent: OAI-SearchBot
Allow: /

User-agent: GPTBot
Disallow: /

This example expresses two decisions: permit OAI-SearchBot to crawl for ChatGPT search features and tell GPTBot not to crawl for model-training use. It does not guarantee that a permitted page will appear in an answer.

Do not add a ChatGPT-User block expecting it to control automatic ChatGPT search. OpenAI directs publishers to use OAI-SearchBot for search opt-outs and notes that robots.txt may not apply to user-initiated ChatGPT-User requests.

When the distinction matters

A documentation site may want search visibility while declining model-training crawling. A support portal may allow both automatic agents but still require authentication for private tickets. A public page can also be allowed in robots.txt yet remain unavailable because the server returns an error, blocks an IP range or serves a restrictive page-level directive.

Robots.txt describes crawl preference; it is not authorization and it is not the only signal involved.

Common mistakes

• Assuming one OpenAI user-agent controls every OpenAI product.
• Blocking GPTBot and expecting removal from ChatGPT search.
• Allowing OAI-SearchBot and describing citation as guaranteed.
• Treating ChatGPT-User as an always-on index crawler.
• Copying a policy to a subdomain without verifying that subdomain’s own /robots.txt.
• Ignoring cached robots responses after a deployment.

Check the policy with Scopeva

1. Run the AI Crawler Checker on the production URL.
2. Confirm that OAI-SearchBot, GPTBot and ChatGPT-User appear as separate results.
3. Use the Robots.txt Tester for representative paths.
4. Compare matched rules with Scopeva’s robots evaluation model.
5. Read the result states and limitations before changing production policy.

For a cross-provider policy, continue with How to configure robots.txt for AI crawlers. For broader context, read What are AI crawlers?.