robots.txt blocks PerplexityBot (Perplexity live search)
If this finding appears in a report, a sampled robots.txt rule was interpreted as disallowing PerplexityBot for the tested path. This reference page does not test your site. Perplexity documents this bot for search discovery, separately from its Perplexity-User fetcher for user-requested visits.
Why this matters
An unwanted restriction can limit search crawling, but an intentional restriction may be appropriate. Perplexity says its user-requested fetcher generally ignores robots.txt, so this finding does not prove that every Perplexity access route is blocked. Allowing a crawler does not guarantee recommendations, citations or human visits.
How to fix it
Agree the intended public paths before editing the complete matching robots.txt groups. Preserve deliberate exclusions, training-policy choices and private-service authentication. If the search restriction is unwanted, change that rule and verify effective path matching; do not assume line order decides access or remove all restrictions automatically. Fetch the deployed file and affected public pages, then inspect verified crawler requests and response codes. If the host demonstrates a firewall block, investigate that specific rule using the provider's current identity guidance. Keep security protections in place. A successful ordinary fetch tests delivery, not whether the provider has recrawled or cited the page.
Turn website findings into an action plan
Create an account to start a Full Audit of your website, with supporting evidence and a prioritised action plan.
Start Full Audit