robots.txt declares a Crawl-delay directive
Crawl-delay is an instruction in robots.txt that asks crawlers to wait a set number of seconds between requests to your site, originally meant to protect servers from being overwhelmed by traffic. This check found a Crawl-delay directive present in your robots.txt. Google ignores Crawl-delay completely, so it has no effect on Google or Google AI Overviews either way. Bing and Yandex do honour it, but the trade-off is slower indexing of new or updated content on those platforms, which is rarely worth it unless your server genuinely struggles under crawler traffic. Your developer needs about 10 minutes. Remove the Crawl-delay line from robots.txt unless you have a specific, current server-load reason to keep it. For real control over crawl rate, use Google Search Console and Bing Webmaster Tools instead, which offer more precise per-platform controls than a blanket delay.
Why this matters
Google ignores Crawl-delay completely, so it has no effect on Google or Google AI Overviews either way. Bing and Yandex do honour it, but the trade-off is slower indexing of new or updated content on those platforms, which is rarely worth it unless your server genuinely struggles under crawler traffic.
How to fix it
Your developer needs about 10 minutes. Remove the Crawl-delay line from robots.txt unless you have a specific, current server-load reason to keep it. For real control over crawl rate, use Google Search Console and Bing Webmaster Tools instead, which offer more precise per-platform controls than a blanket delay.