robots.txt blocks OAI-SearchBot (ChatGPT live search - separate from GPTBot)
If this finding appears in a report, the sampled robots.txt was interpreted as disallowing OAI-SearchBot for the tested path. This reference page does not inspect your website. OAI-SearchBot supports ChatGPT search; GPTBot is a separate training crawler with an independent robots.txt setting.
Why this matters
OpenAI says opting out of OAI-SearchBot excludes content from its search answers, although a navigational link may still appear. Allowing search access does not require allowing training. Permission alone does not guarantee a crawl, citation or referred visitor.
How to fix it
Confirm which public paths the owner intends to make available. Inspect the complete matching crawler groups and path rules; moving an Allow line earlier is not the priority mechanism. Matching groups can combine, and the most specific matching path rule controls. Preserve intended private-path exclusions and authentication rather than adding a blanket Allow to clear a finding. Change only an unwanted restriction, then verify the deployed robots.txt and affected page responses. Check provider-verified requests and status codes at the actual hosting layer. Investigate a demonstrated firewall denial narrowly; do not disable protections or trust a user-agent string alone. Measure referrals separately from crawler requests.
Turn website findings into an action plan
Create an account to start a Full Audit of your website, with supporting evidence and a prioritised action plan.
Start Full Audit