How Claude chooses which sources to cite
Claude searches Anthropic's own index, cites the fewest sources of any major assistant, and often names a brand without linking it at all.
Claude chooses sources by running a web search against Anthropic's own retrieval index, separate from Google's or Bing's, and citing a source only when it is confident the passage directly supports the claim. That conservatism is why it cites fewer sources per answer than any other major assistant, even while it names brands freely from what it already knows.
That gap between naming and citing is the detail most guides to AI search optimization miss when they treat "getting mentioned by Claude" and "getting cited by Claude" as the same goal. They are not, and the second one is considerably harder to earn.
Why does Claude cite fewer sources than everyone else?
Because it is built to cite conservatively rather than broadly. Anthropic's own documentation describes the web search tool as returning results Claude then reasons over, deciding for itself which passages are worth quoting rather than surfacing everything the search step returns [1].
30-point gap
That gap is worth sitting with. It means a brand can show up constantly in Claude's answers, drawn from what the model already learned in training, while almost never appearing as a clickable citation a reader could verify — visibility without traffic, and without the trust signal a citation carries.
For a company trying to measure its own AI visibility, this makes "was I mentioned" a misleading question to ask about Claude on its own. The more useful pair of numbers is mention rate and citation rate tracked separately, because a healthy mention rate can sit next to a near-zero citation rate without either number telling you so, and only the second one reflects a source Claude was actually willing to point a reader at.
Which crawlers actually control this?
Three, and they are not interchangeable. Anthropic runs a dedicated search-indexing crawler, a live-fetch crawler triggered by a specific conversation, and a separate training crawler, and Anthropic's own support documentation is explicit that blocking the wrong one produces a different, silent failure than blocking another [2].
| Crawler | Job | Effect of blocking it |
|---|---|---|
| Claude-SearchBot | Builds the index behind Claude's web search | You stop being findable through search-triggered citations |
| Claude-User | Fetches a specific page live during a conversation | That one conversation cannot read or quote your page at all |
| ClaudeBot | Collects training data for future models | No effect on today's citations, only on future model knowledge |
How does this compare with how Google's systems decide?
Differently in one important way: Claude's decision to cite happens inside a single retrieval pass Anthropic controls end to end, while how AI Overviews choose sources depends on a fan-out across several sub-queries that behaves more like ordinary Search underneath. A page tuned for one is not automatically tuned for the other.
Anthropic also publishes its own robots.txt control for AI training, separate from its search crawlers, which works on the same principle as Google-Extended: a publisher can permit search and citation while limiting training use, and the two decisions do not have to move together. Treating "block AI" as one on-off switch, rather than a set of separate choices per crawler and per engine, is the single most common way a site loses citations it never meant to lose.
What should a page do to earn one of the scarce citations?
State the claim once, plainly, with nothing hedging around it, because Claude's conservatism means an ambiguous or heavily qualified sentence is more likely to be summarised in Claude's own words than quoted directly. A number with a named source beside it is far easier to trust than a number alone, and a page that states its own bias openly when it recommends itself reads as more trustworthy, not less.
Third-party presence matters more here than almost anywhere else, because Claude's own conservatism about first-party claims makes an independent, already-trusted source disproportionately valuable. Because Claude will happily name a brand from training data with no citation at all, the fastest way to convert a bare mention into a linked citation is often a review platform or comparison page Claude already trusts enough to quote, rather than a rewrite of your own site. That third-party work, alongside the crawler and page fixes above, is what our Claude optimization service is built to run end to end.
Questions people also ask
Does Claude use Google or Bing to find sources?
No. Claude's web search runs against Anthropic's own retrieval system, separate from Google's or Bing's index, so ranking well in either of those does not directly put you in front of Claude's search step.
Why does Claude mention my brand but not link to my site?
Claude names brands it is confident about from its training data far more often than it cites a live source for that claim. The gap between mention and citation is the widest of any major assistant, so being named is common and being linked is comparatively rare.
Which Anthropic crawlers do I need to allow?
Claude-SearchBot builds the index behind live web search, Claude-User fetches a specific page when a conversation calls for it, and ClaudeBot collects training data. The first two govern whether you can be cited today; the third affects only future models.
Is it worth trying to get cited by Claude specifically?
Yes, because it is the hardest engine to enter and the citations it does give are correspondingly durable — Claude is conservative about which sources it trusts, so a citation there tends to reflect a page that has actually earned it.
Sources
- Web search tool — Anthropic (checked 23 September 2026)
- Does Anthropic crawl data from the web, and how can site owners block the crawler? — Anthropic (checked 23 September 2026)
- I tracked 13,184 AI citations across 4 LLMs: here's what I learned — Orbit Media Studios (checked 23 September 2026)
Find out whether Claude is naming you without ever citing you
Book a free 30-minute consultation with Ibrahem and Jeremy. We look at your site while we talk, and you leave with a prioritised plan.
Who wrote this
Jeremy ZavaCo-founder
Leads growth and client partnerships.
Works on: CRO, SEO
Reviewed before publishing by Ibrahem Helal, Co-founder.
- AI search optimization: the 2026 guideHow to get cited by ChatGPT, Gemini, Perplexity, Claude, Copilot and Google's AI features: crawler access, citable pages, third-party proof and measurement.
- How Google AI Overviews choose sourcesAI Overviews break your query into sub-questions, then cite whichever pages answer each part cleanly — length and top-10 rank barely matter.
- How Perplexity picks and ranks its sourcesPerplexity crawls its own index, fetches pages live, and cites more sources per answer than any rival — with the closest overlap to Google's rankings.
- Google-Extended explainedGoogle-Extended is a robots.txt token that controls Gemini training, not a crawler. Blocking it does not remove you from Search or AI Overviews.