PerplexityBot

Questions about PerplexityBot

What is PerplexityBot?

PerplexityBot surfaces and links websites in Perplexity search results. Perplexity says it is not used to crawl content for AI foundation models.

Is PerplexityBot the same as Perplexity-User?

No. Perplexity-User fetches pages when a user asks a question, and Perplexity says it generally ignores robots.txt rules because the fetch was user-requested.

PerplexityBot crawls and indexes web content that Perplexity’s search product may cite when answering user queries. Perplexity’s own documentation states this crawler respects robots.txt rules, and recommends allowing PerplexityBot in your site’s robots.txt file if you want your content to be discoverable in Perplexity’s search results.

Site-owner controls

Disallowing PerplexityBot removes this content from consideration when Perplexity’s search product cites sources in its AI-generated answers — and, unlike Perplexity-User, this rule is actually honoured, since Perplexity’s documentation confirms PerplexityBot respects robots.txt. It does not affect Perplexity-User, the separate user-triggered fetcher Perplexity documents, which generally ignores robots.txt and operates independently of indexing decisions. See /limitations for what a robots.txt rule can and cannot guarantee.

Related crawler

Perplexity also documents Perplexity-User, a separate user-triggered crawler that fetches a specific page in direct response to a user’s request inside a Perplexity product. CrawlPact reports these as distinct rows in the crawler matrix because they serve different purposes.

How to verify PerplexityBot requests

Perplexity AI documents this user agent:

                  Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; PerplexityBot/1.0; +https://perplexity.ai/perplexitybot)
                

Perplexity publishes PerplexityBot's IP ranges at https://www.perplexity.com/perplexitybot.json. Operator documentation.

A user-agent string alone can be copied by anyone; confirming a request really came from Perplexity AI needs your server or CDN logs. CrawlPact audits declared policy and does not see your traffic.

Example robots.txt configuration

To disallow PerplexityBot specifically, without affecting any other crawler:

User-agent: PerplexityBot
Disallow: /

If no dedicated User-agent: PerplexityBot group exists in a domain'srobots.txt, this crawler falls back to whatever the wildcard User-agent: * group says (RFC 9309) — see robots.txt syntax basics for how group selection works. A robots.txt rule is a published instruction; it does not technically prevent access.

Official source

https://docs.perplexity.ai/guides/bots

Verified against the source above as of 8 September 2026 — see how CrawlPact verifies crawler information.