The same conversation keeps happening. Someone shows us the output of their LLM SEO retainer: rewritten H1s, new FAQ blocks, "answer-first" opening paragraphs, a list of prompts they want to show up for. It looks like work. It reads well.
Then we fetch one of their product pages the way a crawler does, and it comes back as a nav bar wrapped around an empty `<div>`. Client-rendered app. Every optimized sentence arrives only after a browser executes JavaScript, and per Vercel's crawler research (November 2024 data), no major AI crawler executes JavaScript. OpenAI's, Anthropic's, Meta's and Perplexity's crawlers fetch the JS files and never run them.
So the copy was polished for an audience that never saw it. Nobody in the engagement thought to check.
That's most of this category right now: content people arguing about phrasing on pages the models can't read.
Nobody knows the citation algorithm, including the people selling it
No AI vendor publishes how an assistant picks which sources to cite. No documentation, no "citation quality score", no leaked weighting. The confident lists of AI ranking factors circulating on LinkedIn are invented.
What is documented is narrower, and more useful:
- You can be excluded outright. OpenAI's docs are explicit that sites opted out of `OAI-SearchBot` "will not be shown in ChatGPT search answers." That's a door, not a dial.
- Retrieval and training are separate jobs. Fastly's May 2026 data splits AI bot traffic into roughly 85% crawlers and 15% fetchers, the fetchers being the ones pulling a page because a user asked something right now. Among those fetchers, ChatGPT accounted for 98%.
- Google says the work is still SEO. Their AI optimization guide puts it plainly: "optimizing for generative AI search is optimizing for the search experience, and thus still SEO." There's no secret second discipline.
The crawl math is brutal. Cloudflare's July 2025 figures put Anthropic's crawl-to-referral ratio at 38,066:1, OpenAI at 1,091:1, Perplexity at 195:1, against Google's 5.4:1. (Caveat: the Claude app sends no referrer, so Anthropic's number is inflated.) These models read enormously and send back very little.
So the only part of the pipeline you control is whether you're readable when the fetch happens. That's the whole game right now, and it's engineering before it's copywriting.
LLM SEO is a machine-readability problem first
Here's the shortest useful test in this entire article. Run your most important page through Jina Reader by prefixing the URL with `https://r.jina.ai/`. It gives you the page as plain text, which is roughly what a fetcher gets.
If the result is a nav bar, a cookie notice and nothing else, stop reading about prompts. You have a rendering bug, not a content strategy problem.
This failure is common enough that Adobe reported 80% of early-access customers for their LLM Optimizer product had critical content-visibility gaps. Four out of five, among companies big enough to get into an enterprise early-access program.
The rest of the picture is equally unglamorous:
| What the data says | Source and date |
|---|---|
| No major AI crawler executes JavaScript | Vercel, Nov 2024 data |
| AI crawlers burn ~1/3 of requests on 404s (ChatGPT 34.8%, Claude 34.2%) vs Googlebot's 8.2% | Vercel, Nov 2024 data |
| >51% of AI requests hit your origin, vs <9% of human requests | Fastly, Jan–May 2026 |
That last one deserves a second look. Your CDN absorbs most human traffic and almost none of the AI traffic, so whatever your origin serves uncached is what the models see. If that response is a shell and the content arrives later, the content doesn't exist.
The 404 rate says something too: these crawlers guess at URLs and get them wrong four times as often as Googlebot, so broken links and unstable URLs cost you more here than in search. For who's fetching what, we covered the bots individually in our guide to AI crawlers.
The engineering layer of LLM SEO
Five things, roughly in order of how much evidence supports them.
Server-render anything you want quoted
Not the whole app. The content. Headings, body copy, prices, specs, comparison tables, FAQ answers. If it matters for citation, it belongs in the HTML the server sends.
To be clear about sourcing: this argument comes from Vercel's crawler data, not from Google, which has never told anyone to server-render for AI. But if the fetchers don't run JavaScript, the conclusion is hard to dodge.
Give the page a skeleton
Vercel's Agent Readability Spec (March 2026) is the most concrete published checklist I've seen, and most of it is 2010-era hygiene:
- Returns 200, with at most one redirect
- Canonical tag present
- Meta description of 50+ characters
- `html lang` set, Open Graph tags present
- Three or more headings, in an actual hierarchy
- Text-to-HTML ratio above 15%
That last one quietly fails a lot of modern sites. A page that's 40KB of copy inside 900KB of framework markup is technically readable and practically noise.
Schema helps, and it isn't a cheat code
Add JSON-LD. Include `dateModified` and a `BreadcrumbList`, both of which Vercel's spec calls for. It makes entities and dates unambiguous, and unambiguous is good.
But don't let anyone sell you schema as the LLM SEO lever. Google's own guidance says structured data "isn't required for generative AI search." It's a clarity aid, not a ranking mechanism, and if the underlying content isn't in the HTML, marking it up changes nothing.
Don't lock the door
Every vendor runs three classes of bot, and they need different answers:
| Bot class | Example | What happens if you block it |
|---|---|---|
| Search/citation bot | OAI-SearchBot, Claude-SearchBot, PerplexityBot | You disappear from answers |
| Training crawler | GPTBot, ClaudeBot | You keep your content out of training sets |
| User-initiated agent | ChatGPT-User, Perplexity-User | Documented as possibly ignoring robots.txt anyway |
Blanket-blocking all of them is how you become invisible while believing you've protected your content. Google's John Mueller, reported by Search Engine Journal in June 2026, put the priority bluntly: "Don't block agents. I think that hurdle will be the biggest, for most sites."
Also worth knowing: TollBit's H1 2026 publisher data found roughly 15% of AI scrapes ignored an active robots.txt disallow, with `ChatGPT-User` at 54%. Robots.txt is a stated preference. Your WAF or CDN is the actual gate.
Keep the page stable and operable
Google's web.dev guide on agent-friendly sites (updated April 2026) says agents perceive your page three ways: screenshots, raw HTML, and the accessibility tree, which it describes as "a high-fidelity map that ignores the visual 'noise' of CSS."
Their asks are practical. Stable layout, because a button that moves between categories confuses an agent. No invisible overlays over controls. Real `<button>` and `<a>` instead of clickable divs. `<label for>` on inputs. Interactive elements bigger than 8 square pixels. And some relief: Google explicitly says "don't worry about perfect code."
You can score this yourself. Lighthouse 13.3 (May 2026) added an Agentic Browsing category, which on standard Chrome comes down to three checks: accessibility tree, cumulative layout shift, and llms.txt format. We ran it on August 26, 2026: Stripe gets 3/3, Shopify 2/3, Airbnb 1/2. Our own site scored 2/3 on the first run, and I'm not going to pretend otherwise; fixing our llms.txt formatting got it to 3/3 the same week. The details of what it measures are in our breakdown of the Lighthouse agentic browsing audit.
The content layer, once the page is actually readable
Now the writing matters. Flagging clearly: this section is my judgment, not data. Nobody has published a study on what makes a passage quotable, and I'd distrust anyone who claims otherwise.
What I'd bet on, based on how retrieval works: answer the question near the top, in a paragraph that survives being lifted out of context. Name the entity instead of saying "we" and "our platform". Put dates on claims. Keep one idea per paragraph so a chunk boundary doesn't slice your point in half. Use tables for anything comparative, because tables are the only formatting that reliably survives text extraction.
What I wouldn't spend time on: llms.txt as a visibility play. Ahrefs studied 137,210 domains in May 2026 and found 97% of published llms.txt files received zero traffic, concluding the file is "largely decoration" for AI visibility. Google says Search ignores it entirely. There are good reasons to publish one anyway, and we went through them in our look at whether llms.txt actually works.
A prioritized LLM SEO checklist
Ordered by strength of evidence, not by how impressive it sounds in a proposal:
| Priority | Work | Why it's here |
|---|---|---|
| 1 | Server-render the content that matters | Fetchers don't run JS (Vercel) |
| 2 | Fix the accessibility tree: semantic elements, labels, ARIA | Google audits it directly; Shopify and Airbnb both fail |
| 3 | Kill layout shift | Named in Lighthouse and Google's agent guide |
| 4 | Per-bot robots and WAF rules | Blocking search bots removes you from answers |
| 5 | Page skeleton: status codes, canonicals, headings, text ratio | Vercel's published spec |
| 6 | Form semantics and clear labels | Prerequisite for agents completing tasks |
| 7 | JSON-LD with dates and breadcrumbs | Useful, explicitly not required |
| 8 | Markdown mirrors and content negotiation | Rare enough to be a real differentiator |
| 9 | llms.txt, format-correct or skipped | Malformed scores worse than absent |
Notice that items 1 through 6 are all things a front-end team does, and none of them involve writing a single new sentence.
The upside is that none of this work is speculative. Google's closing line in the agent-friendly guide is the best sentence anyone has written on this topic: "Everything we suggest to make a site 'agent-ready' also makes sites better for humans." Semantic markup, stable layouts and server-rendered content were already good practice, which is roughly the argument we made about UX fundamentals in app development years before agents existed.
And the traffic is worth having. Shopify's Q1 2026 platform data shows AI-referred shoppers converting about 50% higher than organic, with 14% higher order values. Small volume, high intent.
Where to start
Two commands, ten minutes:
- `https://r.jina.ai/your-most-important-url` and read what comes back.
- `npx lighthouse your-url --only-categories=agentic-browsing`
If both look fine, congratulations, you've earned the right to argue about phrasing. If they don't, you now know why your LLM SEO retainer isn't producing citations.
Or let us run it. Our free GEO audit fetches your pages the way the crawlers do, asks the engines what they say about you today, reviews your bot rules, and sends back the prioritized list. No pitch deck, just findings.




