Start with the fundamentals Google confirms
Most of what makes a website LLM-friendly is classic technical SEO done well. Google's official AI-features guidance says the same foundational SEO practices apply to AI experiences in Search, that it recommends unique people-first content, and that no special "AI text files" or special schema are required. The citation data points the same way. Semrush's study of 5,000 queries and 150,000 citations found Perplexity cited a domain from Google's top ten results in 91% of cases and the exact URL in 82%; for Google AI Overviews the figures were 86% and 67%. An Ahrefs test of 3,311 head terms found much lower ChatGPT overlap — 31.8% at the domain level and 10% at the exact URL — and a 2026 arXiv preprint (still under review) covering 55,393 queries found about 30% of cited domains were not on the first page of search results, so rankings are not the whole story. Crawlability and clean extraction gate everything: LLMs cannot cite what they cannot fetch and parse.
Server-render your key pages
Client-side-only rendering is the single most common reason brands are invisible to LLMs. Many AI retrievers execute limited or no JavaScript, so a page that assembles its content in the browser can look empty to the systems deciding what to cite. Server-render or statically generate your homepage, category pages, product and service pages, and long-form content. A quick test: fetch a key URL with curl or view the raw page source — if your pricing, headings, and answers are not present in the initial HTML, fix that before anything else on this checklist.
Use semantic HTML
Prefer <article>, <section>, <nav>, <header>, <footer>, and <main> with a clean H1-H2-H3 hierarchy over generic <div> soup. Semantic structure helps retrievers segment your page, identify the primary answer block, and lift a self-contained passage into a generated response. Put the direct answer in the first paragraph under each heading rather than burying it after preamble.
Add JSON-LD structured data
Google is explicit that no special schema is required for its AI features, so treat structured data as entity clarification rather than an AI hack. It remains the cheapest way to state unambiguously who you are, what a page is about, and how entities relate. At minimum, ship Organization on every page, Article on long-form content, FAQPage on FAQ pages, and Product or SoftwareApplication on product pages. Validate with Google's Rich Results Test.
Configure robots.txt for AI crawlers
Explicitly allow the crawlers behind the engines you want to appear in. Blocking a crawler reduces your presence in that engine rather than guaranteeing removal - vendor documentation notes opted-out pages can still surface as links, and Google-Extended does not affect Google Search inclusion - so block only for a deliberate licensing or intellectual-property reason. A single company can operate several agents - training crawlers, search-index crawlers, and user-triggered fetchers - and you generally want to allow all of them. The user agents worth knowing:
# robots.txt — allow everyone by default, name AI crawlers explicitly
User-agent: *
Allow: /
User-agent: GPTBot
Allow: /
User-agent: OAI-SearchBot
Allow: /
User-agent: ChatGPT-User
Allow: /
User-agent: PerplexityBot
Allow: /
User-agent: Perplexity-User
Allow: /
User-agent: ClaudeBot
Allow: /
User-agent: anthropic-ai
Allow: /
User-agent: Google-Extended
Allow: /
User-agent: Bingbot
Allow: /
User-agent: Applebot-Extended
Allow: /
User-agent: CCBot
Allow: /
Sitemap: https://www.example.com/sitemap.xml- GPTBot — OpenAI (model training)
- OAI-SearchBot — OpenAI (ChatGPT search)
- ChatGPT-User — OpenAI (fetches pages when a user's request triggers browsing)
- PerplexityBot and Perplexity-User — Perplexity
- ClaudeBot and anthropic-ai — Anthropic / Claude
- Google-Extended — controls Google AI model use of your content (Gemini)
- Bingbot — Bing search and Microsoft Copilot
- Applebot-Extended — controls Apple AI model use of your content
- CCBot — Common Crawl, a dataset many models train on
Publish llms.txt — with honest expectations
llms.txt is a proposed convention: a markdown file at your site root listing your most important URLs with one-line descriptions, giving language models a curated index instead of forcing them to infer your site structure. Be clear-eyed about what it is not. Google states that no special AI files are required for visibility in its AI features, so llms.txt is not a Google visibility signal and will not move AI Overviews on its own. It serves other consumers — AI tools, agents, and assistants that choose to fetch it — and it takes minutes to create. Treat it as a low-cost hedge, not a requirement, and keep it synchronized with your live pages. The example below is illustrative:
# Acme Analytics
> Acme Analytics is a product-analytics platform for SaaS teams. This file indexes our most important pages for LLMs and AI agents.
## Product
- [Features](https://www.example.com/features): Event tracking, funnels, retention, and dashboards
- [Pricing](https://www.example.com/pricing): Current plans, limits, and billing options
- [Integrations](https://www.example.com/integrations): Native connectors and API access
## Docs
- [Quickstart](https://www.example.com/docs/quickstart): Install the SDK and send your first event
- [API Reference](https://www.example.com/docs/api): REST endpoints, authentication, and rate limits
## Comparisons
- [Acme vs. Competitor A](https://www.example.com/compare/competitor-a): Feature and pricing comparison
- [Alternatives](https://www.example.com/alternatives): How Acme compares to other analytics tools
## Company
- [About](https://www.example.com/about): Team and contact details
- [Blog](https://www.example.com/blog): Product updates and original researchCanonical URLs, speed, and stability
Set canonical tags on every page and avoid duplicating content across parameterized URLs — duplicate parameterized URLs can split signals and make it unclear which URL to cite. Keep URL structures shallow, descriptive, and permanent — a cited URL that later 404s is a lost citation. Fast time-to-first-byte, no soft 404s, and reliable uptime matter because retrieval systems deprioritize slow or flaky origins. Add visible last-updated dates and author bylines so both humans and machines can assess freshness and accountability.
The complete checklist
Work through these items in order; the earlier items unblock everything after them.
- Server-render or statically generate all key pages
- Semantic HTML with a clean heading hierarchy and answer-first paragraphs
- JSON-LD (Organization, Article, FAQPage, Product) on every template
- AI crawlers explicitly allowed in robots.txt
- Optional /llms.txt at the root, kept in sync with the site
- Canonical URLs everywhere; no duplicate parameterized content
- Fast TTFB, no soft 404s, stable URLs
- Visible last-updated dates and author bylines
- Monitor AI Overviews and AI Mode performance in Search Console where available