robots.txt Content-Signal directive blocks AI use of your content
Content-Signal is a newer directive added to robots.txt that tells crawlers exactly how they're allowed to use your content, separately from whether they can crawl it at all: for search, for live AI answers, or for training. This check found your robots.txt setting ai-input=no and/or ai-train=no, which explicitly tells AI systems not to use your pages in live answers or in training data. ai-input=no specifically tells AI answer engines like Google AI Overviews, ChatGPT and Perplexity not to draw on your page when generating a live response to a user's question. Cloudflare now applies this directive by default on many sites and is moving toward blocking it automatically on ad-monetised domains, so plenty of businesses carry this setting without ever having chosen it. Your developer needs about 10 minutes. Check the Content-Signal line in robots.txt and, if AI visibility is a goal for your business, change ai-input and ai-train to allow (or remove the restrictive values) so AI systems can use and cite your content.
Why this matters
ai-input=no specifically tells AI answer engines like Google AI Overviews, ChatGPT and Perplexity not to draw on your page when generating a live response to a user's question. Cloudflare now applies this directive by default on many sites and is moving toward blocking it automatically on ad-monetised domains, so plenty of businesses carry this setting without ever having chosen it.
How to fix it
Your developer needs about 10 minutes. Check the Content-Signal line in robots.txt and, if AI visibility is a goal for your business, change ai-input and ai-train to allow (or remove the restrictive values) so AI systems can use and cite your content.