Does Perplexity AI Index Your Website?

Yes, Perplexity AI crawls and indexes websites directly. It runs its own web crawler called PerplexityBot, separate from Google or Bing. Your robots.txt controls whether it gets in. If you have never checked, there is a good chance it is crawling you right now.

The short answer: Perplexity uses its own crawler plus the Bing index as a fallback. To appear in Perplexity citations, your pages need to be crawlable, structured clearly, and answer questions directly. Blocking PerplexityBot keeps you out entirely.

How Perplexity Crawls the Web

Perplexity operates PerplexityBot, a standard HTTP crawler that follows the robots exclusion protocol. It identifies itself with the user-agent string PerplexityBot. If your robots.txt does not specifically block it, it crawls freely.

The crawl is not comprehensive in the way Googlebot is. Perplexity prioritizes pages that answer questions, pages that rank in Bing, and pages that get cited in its existing answers. A page that sits outside those signals can go months without a visit from PerplexityBot.

Perplexity also uses the Bing index as a supplementary source. When its own crawler has not seen a page recently, it falls back to Bing's indexed version of that page. Being indexed by Bing is therefore a secondary path into Perplexity results, even if PerplexityBot has not visited directly.

How to Check If Perplexity Is Indexing You

Three methods, in order of reliability:

1. Search your domain in Perplexity. Open Perplexity.ai and search site:yourdomain.com. If pages appear as cited sources in the answer, Perplexity has indexed them. This is the fastest check.
2. Check your server logs for PerplexityBot. Search your access logs for PerplexityBot in the user-agent field. If it appears, the crawler has visited. The frequency tells you how often it comes back.
3. Search a topic you cover and watch the citations. Search a question your site answers. If competitor pages appear as citations and yours do not, you are either blocked, not indexed, or not structured clearly enough to be selected.

There is no Perplexity equivalent of Google Search Console. You cannot submit a sitemap to Perplexity directly. Crawl discovery is passive: it finds you through links, through the Bing index, or through its own crawl schedule.

What Can Block PerplexityBot

Four things stop Perplexity from indexing your pages:

1. robots.txt disallow. A Disallow: / rule under User-agent: * blocks all crawlers including PerplexityBot. A specific User-agent: PerplexityBot block does the same with surgical precision.
2. Noindex meta tag. <meta name="robots" content="noindex"> tells crawlers not to include the page in results. PerplexityBot respects this.
3. JavaScript-rendered content with no SSR. If your page requires JavaScript execution to show its content and you are not server-side rendering, PerplexityBot may not execute the JavaScript. The page appears empty. No content, no citation.
4. Login gates or paywalls. If a page requires authentication, the crawler cannot read it. The page does not get indexed and will not be cited.

Gambling affiliate sites occasionally block all crawlers on their main comparison pages for competitive reasons. This keeps them out of Perplexity entirely, which is a tradeoff worth making consciously rather than by accident.

How Perplexity Selects What to Cite

Being indexed is not the same as being cited. Perplexity indexes many more pages than it cites. Selection for a citation happens at query time. The engine retrieves candidate pages and selects the ones that best answer the specific question.

Pages that get cited share three patterns: they deliver a direct answer early, they use structured formatting that is easy to parse, and they come from domains with external credibility signals. A page that buries its answer in paragraph eight will be crawled but not cited.

Perplexity's citation behavior is more transparent than Google's. The cited sources appear visibly as numbered links in every answer. That visibility makes it easier to study what is being cited for your target queries and reverse-engineer the pattern.

What Gambling Affiliates Should Know

Perplexity does not apply the same gambling content restrictions that Google AI Overviews apply in regulated markets. A search for casino bonuses or wagering requirements in Perplexity will surface affiliate content that Google might suppress in an AI Overview for UK or Australian users.

This makes Perplexity worth optimizing for explicitly, not just as a byproduct of Google SEO. The citation surface is smaller than Google, but the content policies are less restrictive for gambling queries.

The practical steps are the same as for any AI citation: structured content, direct answers, no login gate, server-side rendering, and no robots.txt block on PerplexityBot. The only gambling-specific addition is to not assume that Google's content suppression applies equally here. It does not.

Check Your robots.txt Now

Open yourdomain.com/robots.txt. Look for any rule that blocks * or specifically blocks PerplexityBot. If you are blocking it unintentionally, removing the rule is the fastest way to get back into Perplexity's crawl queue.

After that, the structural work is what drives citation. Being crawlable gets you in the pool. Being structured clearly is what gets you selected.