CrawlPact

GPTBot vs. OAI-SearchBot vs. ChatGPT-User: which should you block?

Published 7/24/2026

OpenAI documents three distinct crawler tokens, each triggered by a different thing. Treating them as one “OpenAI bot” and blocking or allowing all three together is the most common misconfiguration CrawlPact sees in this operator’s traffic.

The three tokens

  • GPTBot — training. Crawls content that may inform future model training.
  • OAI-SearchBot — search. Surfaces and links to pages in ChatGPT search results.
  • ChatGPT-User — user-triggered. Fetches a specific page because a person asked ChatGPT a direct question about it.

The decision

  • Disallow GPTBot if you don’t want your content used for OpenAI’s model training. This has no effect on whether your site can be found through ChatGPT search.
  • Leave OAI-SearchBot allowed if you want your pages discoverable through ChatGPT’s search features — disallowing it removes that visibility entirely, similar to disallowing a search engine’s indexer.
  • Leave ChatGPT-User allowed unless you specifically don’t want any content served in response to direct user questions inside ChatGPT — blocking it does not reduce training or search exposure, since it’s a separate, user-initiated request path.

Common mistake CrawlPact flags

A single User-agent: GPTBot group is sometimes assumed to cover “all of OpenAI.” It doesn’t — robots.txt groups match on the exact token named, with no fallback to a shared vendor identity. Each token needs its own explicit Disallow if you want to restrict it. CrawlPact’s AI crawler checker shows the result for all three, evaluated independently, so a gap is visible immediately.

See how this applies to your own site

Run a free audit to check your declared AI crawler policy against your own domain.

Audit a domain

Analytics preferences

CrawlPact uses optional Google Analytics and Microsoft Clarity on public marketing pages to understand which content is useful and how visitors actually use it. Neither is used in the authenticated app or admin areas. You can accept or decline analytics without affecting the service.