Google-Extended vs. Googlebot: what each one actually controls

Google operates Googlebot for Search indexing and Google-Extended as a separate opt-out token for generative AI training use (Gemini, Vertex AI). These are frequently confused because they come from the same operator and are often configured in the same robots.txt file.

Should I block Google-Extended or Googlebot?

To opt out of Gemini training and grounding without affecting Google Search, disallow Google-Extended and leave Googlebot allowed. Google says Google-Extended does not affect Search inclusion or ranking. AI Overviews and AI Mode are part of Search: they follow Googlebot’s rules, Google’s snippet controls and the Search generative AI control in Search Console described below.

What happens when you block each one?

• Disallow Googlebot only if you want to leave Google Search entirely — this is rarely the right choice for a public website that depends on organic search traffic.
• Disallow Google-Extended if you want to opt specific content out of generative AI training while keeping full Search visibility.

A third Google lever: the Search generative AI control

Google-Extended does not decide whether your site appears in AI Overviews or AI Mode — Google documents that it “does not impact a site’s inclusion in Google Search.” Short of blocking Googlebot and leaving Search entirely, that choice has, since 31 August 2026, its own switch: the Search generative AI control in Search Console, which includes or excludes a site from AI Overviews, AI Mode and Discover’s generative AI features. It is set in Search Console, not robots.txt, so CrawlPact cannot see it in an audit. See what Google’s generative AI performance reports show for how to measure that visibility.

Why do “block all AI” snippets block Google Search?

Copying a “block all AI” robots.txt snippet from a general audience article often disallows Googlebot by accident, alongside AI-specific tokens. CrawlPact’s conflict detector raises this as a high-severity finding when a preset that expects search visibility is combined with a rule that blocks Googlebot.

Check which of the two your own site currently allows or blocks with the AI crawler checker.