Skip to content

robots.txt blocks GPTBot (OpenAI training opt-out)

GPTBot is OpenAI's crawler for content that may be used to train generative AI foundation models. OpenAI documents GPTBot separately from OAI-SearchBot, which is the crawler used for ChatGPT search. This check found a robots.txt rule disallowing GPTBot.

Why this matters

This is primarily a model-training policy choice, not a current ChatGPT Search visibility failure. OpenAI says the GPTBot and OAI-SearchBot settings are independent. Blocking GPTBot indicates that the site's content should not be used for training OpenAI's generative AI foundation models; it does not by itself opt the site out of ChatGPT search.

How to fix it

No change is required if the training opt-out is deliberate. If you want to permit potential use of the site's content in OpenAI model training, remove or narrow the GPTBot Disallow rule. Manage ChatGPT Search visibility separately with OAI-SearchBot.

Turn website findings into an action plan

Create an account to start a Full Audit of your website, with supporting evidence and a prioritised action plan.

Start Full Audit