CrawlPactOpenAI · Search crawler
OAI-SearchBot
Used to discover and surface links to websites in ChatGPT search results.
Questions about OAI-SearchBot
What is OAI-SearchBot?
OAI-SearchBot is OpenAI's search crawler. It surfaces websites in ChatGPT's search features.
What happens if I block OAI-SearchBot?
OpenAI says sites that opt out of OAI-SearchBot are not shown in ChatGPT search answers, though they can still appear as navigational links. OpenAI says a robots.txt change can take about 24 hours to be reflected in search.
Does allowing OAI-SearchBot allow AI training?
Training is GPTBot's documented role, and each setting is independent. OpenAI notes that if you allow both GPTBot and OAI-SearchBot, it may use the results of one crawl for both purposes.
OAI-SearchBot is OpenAI’s documented crawler for surfacing and linking to web content inside
ChatGPT’s search features, distinct from the training-focused GPTBot.
Site-owner controls
Disallowing OAI-SearchBot removes this content from ChatGPT’s search-style results. It does
not affect GPTBot (OpenAI’s training crawler) or ChatGPT-User (fetches made in direct
response to a user’s request inside ChatGPT) — both evaluated as separate tokens. See
/limitations for what a robots.txt rule can and cannot guarantee.
Why the distinction matters
A website that wants to remain discoverable through AI-assisted search while still restricting
training use of its content needs to treat OAI-SearchBot and GPTBot as separate decisions.
CrawlPact’s “Allow Search, Block Training” preset is built around exactly this kind of
distinction.
How to verify OAI-SearchBot requests
OpenAI publishes OAI-SearchBot's IP ranges at https://openai.com/searchbot.json. Its user agent includes OAI-SearchBot and a link to https://openai.com/searchbot. Operator documentation.
A user-agent string alone can be copied by anyone; confirming a request really came from OpenAI needs your server or CDN logs. CrawlPact audits declared policy and does not see your traffic.
Example robots.txt configuration
To disallow OAI-SearchBot specifically, without affecting any other crawler:
User-agent: OAI-SearchBot
Disallow: /If no dedicated User-agent: OAI-SearchBot group exists in a domain'srobots.txt, this crawler falls back to whatever the wildcard User-agent: * group says (RFC 9309) — see robots.txt syntax basics for how group selection works. A robots.txt rule is a published instruction; it does not technically prevent access.
Official source
https://developers.openai.com/api/docs/bots
Verified against the source above as of 24 July 2026 — see how CrawlPact verifies crawler information.
Related guides
- Blocking AI training while staying visible in AI searchChoosing between CrawlPact's presets when the goal is opting out of model training without losing AI-search discoverability.
- GPTBot vs. OAI-SearchBot vs. ChatGPT-User: which should you block?OpenAI operates three separate crawler tokens for three separate purposes. A decision guide for choosing which, if any, to disallow.
Other OpenAI crawlers
- ChatGPT-UserFetches a web page in direct response to a user's question inside ChatGPT or a Custom GPT.
- GPTBotUsed by OpenAI to crawl publicly accessible web content that may be used to train future models.
- OAI-AdsBotValidates the safety and relevance of web pages submitted as ads on ChatGPT — not used for AI training.
Other search crawlers
See how this applies to your own site
Run a free audit to check your declared AI crawler policy against your own domain, or use the AI crawler checker to see how your robots.txt treats OAI-SearchBot and every other documented crawler.