Can AI see your website? How to check

Check 1: is your content in the raw HTML?

Right-click your homepage, choose View Page Source, and search for a sentence you know appears in your main content - a product description, a service line, a paragraph from your about page. If you cannot find it in that raw HTML, the browser is building your content afterwards, once JavaScript runs and hydrates the page. Most AI crawlers fetch the raw response and never execute that JavaScript, because running a full browser engine for every page on the web is expensive at the scale these companies operate. Googlebot is the exception - it renders pages through a headless Chrome pipeline before indexing - which is exactly why a JavaScript-heavy site can rank well in classic search while remaining unreadable to ChatGPT, Claude or Perplexity. This is the single most common and most severe AI-visibility failure, because it does not just hide one page - it can hide the entire site behind a near-empty shell. The fix is prerendering your public pages at build time, or rendering them server-side per request, so the HTML that arrives at the crawler already contains the words a customer would read.

Check 2: are AI crawlers allowed in?

Open yourdomain.com/robots.txt in a browser and read through the Disallow lines, checking specifically for GPTBot, ClaudeBot, PerplexityBot, OAI-SearchBot and Google-Extended. Blocking these crawlers was a popular defensive move in 2023, when AI training was the main story and plenty of site owners added a blanket block out of caution, sometimes copied from a template without much thought about what it actually stopped. Many of those entries have never been revisited since, even as the AI landscape has moved from training controversy to something closer to a second search engine that customers actually use to find businesses. Blocking a crawler is a legitimate, defensible choice when you make it deliberately - but a rule sitting untouched in a text file from three years ago is not a decision, it is an accident, and accidents in robots.txt cost you visibility silently, with no error message and no obvious symptom beyond fewer AI-driven enquiries.

Check 3: do you have structured data?

Paste your homepage URL into Google's Rich Results Test or validator.schema.org and see what markup comes back. At minimum you want Organization schema describing who you are; beyond that, FAQ schema on question-and-answer content, Product or Service schema for what you sell, and Review or LocalBusiness schema where relevant all help. Structured data matters here for a specific mechanical reason: retrieval and answer-generation systems have to extract facts from your page before they can use them, and prose is ambiguous in a way that costs the model extra work to resolve - is this address the business's location or a mentioned competitor's? Is this number a price or a phone number? JSON-LD markup answers those questions unambiguously, in a format built for machines rather than inferred from sentence structure. A page with clean, valid schema is simply cheaper and safer for a model to quote correctly, which is exactly the kind of source these systems prefer to lean on when composing an answer.

Check 4: does llms.txt exist?

Visit yourdomain.com/llms.txt directly. Most sites return a 404 today, which is normal - the format is new, unofficial, and not every AI company has confirmed it uses it - but that also makes it an easy, low-effort differentiator while adoption is still shaking out. The idea, borrowed from the logic of robots.txt and sitemap.xml, is a plain markdown file that gives an AI system your canonical summary in a single fetch: who you are, what you offer, and links to your most important pages, written in plain language rather than buried in navigation and marketing copy. Whether or not every model currently reads it, having one costs an afternoon and gives any system that does look for it a clean, unambiguous starting point instead of having to piece your business together from a homepage full of hero banners and calls to action.

Check 5: ask the assistants directly

Ask ChatGPT and Perplexity directly: what does [yourdomain.com] do? A confident, accurate answer means the retrieval pipeline behind that assistant found your site, fetched it successfully, and extracted something usable - the whole chain worked end to end. A refusal, a vague guess, or an answer describing an old version of your business tells you retrieval is failing somewhere along that chain, and the previous four checks are exactly where to look for why: unrendered content, a robots.txt block, missing structured data, or no llms.txt to fall back on. This test is also worth repeating over time rather than treating as a one-off, because these assistants increasingly browse live rather than relying purely on training data, which means a fix you make today can show up in an answer within days rather than waiting for the next model training cycle.

What being invisible actually costs you

None of this is abstract. When someone asks an assistant to recommend a supplier, compare options, or explain what a business does before they visit its website, the assistant can only draw on businesses it can actually read - it cannot recommend what it cannot retrieve. A site that fails these checks does not get a bad mention; it gets no mention at all, quietly excluded from a growing share of buying research that never touches a traditional search results page. The frustrating part is that most of these failures are configuration problems rather than content problems - a good business with genuinely useful content can be sitting behind a JavaScript shell or an old robots.txt block, invisible for reasons that have nothing to do with the quality of what it offers. That also makes them unusually cheap to fix once identified, which is the whole point of running through the five checks above before assuming the problem is something bigger.

The automated version

Running all five checks manually takes about fifteen minutes and tells you where the obvious gaps are. AuditHQ's AI Visibility suite runs these same checks plus the deeper layer behind them - schema depth and validity, your entity's sameAs network linking your business across platforms, whether your page structure is citation-friendly, and whether you carry clear author and freshness signals - in one pass, returning a score and a specific, prioritised list of fixes rather than a raw pass or fail. It is one of nine suites included in the free 60-second scan, so you can see exactly where AI visibility sits alongside the rest of your site's technical health rather than checking it in isolation.

Frequently asked questions