llms.txt Explained: Does It Actually Work
llms.txt is a proposed file that lists a site's key pages for an AI model to read. No major assistant is confirmed to use it yet. Here is what it actually does.
llms.txt is a proposed plain-text file, placed at a site's root, that lists its most important pages in a short, structured summary meant to be easy for a language model to read. No major AI assistant has confirmed that it reads or gives weight to this file, so treat it as unproven rather than as infrastructure.
That distinction matters for prioritizing your time within generative engine optimization, where crawler access and content quality have a far stronger evidence base than this one file does.
What is llms.txt supposed to do?
The idea, as published at llmstxt.org, is a markdown file at /llms.txt that gives a language model a short, curated map of a site: what it is, and links to the pages worth reading, in a format lighter than crawling the whole site and guessing [1]. The proposal's own reasoning is that a full page, with navigation, styling and unrelated content, is a noisy way for a model to learn what a site actually offers, and a hand-curated summary avoids that noise.
It is a reasonable idea on paper. The gap is entirely about adoption: a format only helps if something reads it, and that is the part still unresolved. A sitemap works because every major search engine agreed, years ago, to read it. llms.txt has no equivalent agreement yet, from any of the companies whose models would need to act on it, which is a very different starting position for a new format.
Does any AI assistant actually read it?
Not confirmed. OpenAI's own documentation of its crawlers describes what each one does and does not do, and llms.txt support is not among the behaviors it documents [2]. Anthropic, whose own crawler and web-search behavior is covered in getting cited by Claude, has made no public confirmation either, and neither has Google or Perplexity. Independent testing summarized by industry coverage of the format has likewise found no consistent evidence that major assistants change their answers based on an llms.txt file's presence or content [3].
The same absence of confirmation holds for Google's AI features specifically. AI Overviews vs AI Mode covers how those two surfaces build on Google's existing crawl and ranking systems, and nothing in Google's own documentation names llms.txt as an input to either one.
| llms.txt | robots.txt |
|---|---|
| A proposal, not a standard | A real, decades-old crawler standard |
| No confirmed adoption by any major AI vendor | Explicitly honored by OpenAI, Anthropic, Google, Perplexity |
| Offers a curated summary of key pages | Controls which pages a crawler may access at all |
| Effect on citations is unconfirmed | Directly determines whether a page can be seen |
This is the comparison worth sitting with before spending real time on either. One of these two files is doing confirmed, load-bearing work right now; the other might be doing nothing at all.
Should you publish one anyway?
If it costs an hour and everything else in this cluster — crawler access, structured facts on your pages, third-party mentions — is already handled, a low-priority llms.txt file is a reasonable extra with no real downside. It costs nothing to be wrong about, and if adoption does eventually happen, an early file is a small head start.
What belongs in it, if you write one?
Keep it short: a one-line description of the business, then a handful of links to the pages you would want a model to read first if it only read a few — pricing, your strongest guides, a clear services overview. Do not treat it as a place to stuff keywords or duplicate your sitemap; the entire premise of the format is a curated, minimal list, and padding it defeats the one property that might someday matter.
If you want to know whether any of this is moving the number that matters, measuring AI visibility covers how to track citations over time, which is the only honest way to tell whether an llms.txt file changed anything for your specific site. Our GEO service prioritizes the confirmed work first and treats a file like this as the hour-long extra it is, not the headline deliverable.
أسئلة يطرحها الناس
Is llms.txt an official web standard?
No. It is a proposal from one contributor, published at llmstxt.org, with no confirmation from OpenAI, Anthropic, Google or Perplexity that their models read or prioritize it.
Does adding an llms.txt file hurt anything?
No, it is harmless to publish. The concern is not risk, it is that teams spend the time expecting a payoff no vendor has confirmed exists.
Should I still write one?
If it costs you an hour and you have already done the crawler, schema and content work this cluster covers, it is a reasonable low-priority extra. It is not a substitute for that other work.
What is the difference between llms.txt and robots.txt?
robots.txt is a real, universally honored standard that controls crawler access. llms.txt is an unconfirmed proposal that, at best, offers a curated summary a model may or may not read.
المصادر
- The /llms.txt file — llmstxt.org (checked 23 September 2026)
- Overview of OpenAI crawlers — OpenAI (checked 23 September 2026)
- What is llms.txt and does it help AI search visibility — Semrush (checked 23 September 2026)
Fix what's actually confirmed to affect AI citations
احجز استشارة مجانية مدتها 30 دقيقة مع إبراهيم وجيريمي. ننظر في موقعك أثناء الحديث، وتخرج بخطة مرتّبة حسب الأولوية.
من كتب هذا
Ibrahem HelalCo-founder
Leads strategy and delivery.
يعمل على: SEO, GEO
راجعه قبل النشر Jeremy Zava، Co-founder.
المزيد عن هذا
- Generative engine optimization: the playbookHow answer engines pick what to cite, what a quotable page looks like, which crawlers to allow, how to measure it, and a 90-day plan built on evidence.
- How to Get Cited by ClaudeClaude now searches the web and cites what it finds. Here is what its crawler needs, what kind of page it trusts, and how that differs from ChatGPT.
- AI Overviews vs AI Mode: What's the DifferenceAI Overviews sits inside normal Google results; AI Mode is a separate conversational search. Here is how they differ, and which to optimize for first.
- How to Measure AI VisibilityAI visibility means tracking whether assistants name and link your brand across a fixed set of buyer questions run weekly, not a single good screenshot.