Skip to content

robots.txt blocks Google-Extended (Gemini training and grounding control)

Google-Extended is a robots.txt product token that lets publishers control whether content Google has crawled may be used to train future Gemini models and for grounding in Gemini Apps and Vertex AI with Google Search. It does not control whether the site can appear in Google Search, and Google states that it is not a Search ranking signal. This check found a robots.txt rule disallowing Google-Extended.

Why this matters

This is a publisher policy choice, not a Google Search visibility failure. Blocking Google-Extended limits the ways Google may use crawled content for future Gemini training and certain Gemini or Vertex AI grounding experiences. It does not by itself remove pages from Google Search or make a site invisible to AI Overviews.

How to fix it

Only change this rule if you want to permit the Google-Extended uses described by Google. If the block is deliberate, no fix is required. If you do want to allow it, remove or narrow the Google-Extended Disallow rule in robots.txt and then verify the published file.

Turn website findings into an action plan

Create an account to start a Full Audit of your website, with supporting evidence and a prioritised action plan.

Start Full Audit