Google-Extended: opting out of Bard / Gemini training without losing Google search

When Google announced Google-Extended in September 2023, the framing was deliberately narrow: this is a control for AI products like Bard (now Gemini) and Vertex AI generative APIs. It is not a control for Google Search. Disallowing Google-Extended in robots.txt does not affect Search crawling, indexing, or ranking.

This matters because the conventional User-agent: Googlebot block — which a small number of publishers use to opt out of Search entirely — is much heavier-handed than what most publishers actually want. Google-Extended gives you the choice to be discoverable in Search but not used to train Gemini.

The directive

User-agent: Google-Extended
Disallow: /

Add this to robots.txt at the site root. Google's crawler infrastructure respects it on subsequent fetches; pages already used for training are not retroactively removed (training is a one-time read).

What it covers

Per Google's announcement and ongoing crawler documentation:

• Bard / Gemini generative responses
• Vertex AI Generative APIs that train on web content
• Future "generative APIs" Google launches under the same control

It does not cover:

• Google Search ranking signals
• Google Discover
• Google News
• Featured snippets (extracted from indexed content, not generative)

A common confusion: Googlebot still crawls you

Adding the Google-Extended block does not stop Googlebot from crawling. The two are separate tokens with separate policies. If you want to keep Search visibility but block training, you keep Googlebot allowed and add the Google-Extended block — exactly the pattern publishers usually want.

What this does NOT do for AI Overviews

This is the subtle part. Google AI Overviews (the AI-generated summary panels in Search results) use Search-indexed content, not the Gemini training corpus. So blocking Google-Extended does not remove your domain from AI Overview citations. To opt out of those, you'd need to opt out of Search entirely (block Googlebot), which is a much bigger trade.

In practice this means publishers can opt out of Gemini training without losing AI Overview presence. Whether that's what you want depends on whether you view AI Overviews as positive (extra brand surface) or negative (zero-click summaries that cannibalise traffic).

How AI Visibility Monitor checks this

The Crawler Access tool checks for the Google-Extended directive specifically, separate from the standard Googlebot check. We surface "you've blocked AI training but kept Search visibility" as a positive signal in the Readiness score because it represents an intentional, well-scoped control rather than a blanket block that costs SEO traffic.