Skip to content

robots.txt blocks Perplexity-User (live browsing agent)

Perplexity-User is Perplexity's on-demand crawler, triggered when a real user's search pulls your page into an answer in real time, separate from PerplexityBot, the bulk crawler that indexes content in advance. This check found robots.txt disallowing Perplexity-User specifically. Both crawlers matter, but they cover different moments in a prospect's research. Even if PerplexityBot is allowed to crawl your site broadly, blocking Perplexity-User removes you from live, in-the-moment Perplexity answers, often the exact scenario where a prospect is actively comparing options. This gap can sit unnoticed because the bulk crawler being allowed makes the setup look fine at a glance. Your developer needs about 15 minutes. Allow Perplexity-User in robots.txt unless you specifically intend to block Perplexity's live, user-triggered visits. This is worth fixing even if PerplexityBot is already allowed, since the two crawlers serve genuinely different moments in the buying journey.

Why this matters

Even if PerplexityBot is allowed to crawl your site broadly, blocking Perplexity-User removes you from live, in-the-moment Perplexity answers, often the exact scenario where a prospect is actively comparing options. This gap can sit unnoticed because the bulk crawler being allowed makes the setup look fine at a glance.

How to fix it

Your developer needs about 15 minutes. Allow Perplexity-User in robots.txt unless you specifically intend to block Perplexity's live, user-triggered visits. This is worth fixing even if PerplexityBot is already allowed, since the two crawlers serve genuinely different moments in the buying journey.