Skip to content

robots.txt blocks Google-Extended (Gemini training)

Google-Extended is a specific crawler control that lets Google use your content to train Gemini (its AI model) and to generate AI Overviews (the AI-written summaries that appear above normal search results), separate from regular Googlebot indexing. This check found a robots.txt rule disallowing Google-Extended. Gemini handles a growing share of AI search queries, and Google's AI Overviews now appear across a wide range of ordinary search results too. Blocking Google-Extended removes your content from both pools at once, meaning you can still rank normally in Google Search while being invisible to the AI-generated summaries sitting above those results. Your developer needs about 10 minutes. Remove the Google-Extended Disallow rule for / (the whole site) in robots.txt, unless you have a deliberate reason to opt your content out of Gemini training and AI Overviews. Confirm the change by checking robots.txt directly after publishing, since a cached version can take a little while to clear on some CDNs.

Why this matters

Gemini handles a growing share of AI search queries, and Google's AI Overviews now appear across a wide range of ordinary search results too. Blocking Google-Extended removes your content from both pools at once, meaning you can still rank normally in Google Search while being invisible to the AI-generated summaries sitting above those results.

How to fix it

Your developer needs about 10 minutes. Remove the Google-Extended Disallow rule for / (the whole site) in robots.txt, unless you have a deliberate reason to opt your content out of Gemini training and AI Overviews. Confirm the change by checking robots.txt directly after publishing, since a cached version can take a little while to clear on some CDNs.