robots.txt blocks Meta-ExternalFetcher (Meta AI live fetch)
Meta-ExternalFetcher is Meta's crawler for fetching a single page live when Meta AI needs it to answer a specific question, distinct from Meta-ExternalAgent's broader training crawl. This check found robots.txt disallowing Meta-ExternalFetcher. Meta has stated this particular fetcher may not always respect robots.txt on user-initiated requests, so the finding is more diagnostic than a guaranteed block. If Meta AI visibility matters to your business, blocking this fetcher signals the wrong intent even though Meta says it isn't always honoured, and removing it costs nothing. It's low-severity on its own, but worth clearing up so your robots.txt consistently reflects that you want AI visibility rather than sending mixed signals. Your developer needs about 15 minutes. Allow Meta-ExternalFetcher in robots.txt unless the block is intentional, so the file no longer contradicts your broader AI visibility goals. Since Meta has said this fetcher doesn't always respect robots.txt on live requests anyway, treat this fix as good hygiene rather than a guaranteed unlock.
Why this matters
If Meta AI visibility matters to your business, blocking this fetcher signals the wrong intent even though Meta says it isn't always honoured, and removing it costs nothing. It's low-severity on its own, but worth clearing up so your robots.txt consistently reflects that you want AI visibility rather than sending mixed signals.
How to fix it
Your developer needs about 15 minutes. Allow Meta-ExternalFetcher in robots.txt unless the block is intentional, so the file no longer contradicts your broader AI visibility goals. Since Meta has said this fetcher doesn't always respect robots.txt on live requests anyway, treat this fix as good hygiene rather than a guaranteed unlock.