# Does llms.txt Actually Do Anything in 2026? Here's What the Data Shows
TL;DR: No — not for AI search visibility. Multiple independent 2026 studies covering hundreds of thousands of domains and over 500 million AI bot visits found essentially zero correlation between having an llms.txt file and getting cited by ChatGPT, Claude, or Perplexity, and Google has stated outright that Search ignores the file. It still has one legitimate, narrower use: helping coding assistants like Cursor and Claude Code retrieve documentation efficiently.
What llms.txt Was Supposed to Do
Jeremy Howard, co-founder of Answer.AI and fast.ai, proposed llms.txt in September 2024 as a plain-Markdown file at a site's root (/llms.txt) that summarizes a site's most important pages, stripped of navigation, ads, and JavaScript cruft. The pitch was straightforward: LLM context windows are limited and crawling raw HTML is wasteful and error-prone, so give models a curated index instead — something like a sitemap.xml written for language models instead of search engine crawlers.
A minimal llms.txt looks like this:
# Acme Corp
> Acme Corp builds infrastructure tooling for distributed systems teams.
## Docs
- [Getting Started](https://acme.example.com/docs/getting-started): Install and configure the CLI
- [API Reference](https://acme.example.com/docs/api): Full REST and gRPC endpoint reference
- [Architecture Overview](https://acme.example.com/docs/architecture): How the control plane and agents communicate
## Optional
- [Changelog](https://acme.example.com/changelog): Release notes by version
- [Blog](https://acme.example.com/blog): Engineering posts and case studiesThe idea spread fast through the SEO and content-marketing world in 2025, riding the same wave as "AEO" (answer engine optimization) and "GEO" (generative engine optimization). By 2026, tooling had caught up — Mintlify auto-generates /llms.txt and /llms-full.txt for every docs site it hosts, and a wave of "llms.txt generator" tools appeared to help marketing teams produce one in minutes.
None of that tells you whether any AI system actually reads it. That's a separate, empirical question, and 2026 is the first year there's enough real traffic data to answer it.
The Adoption Numbers
Adoption is genuinely non-trivial at this point. Rankability's June 2026 tracking found 8.7% of the top 1,000 global websites publish an llms.txt. SE Ranking's broader study of roughly 300,000 domains put adoption at 10.13% — with mid-traffic sites (1,001–5,000 monthly visits) actually adopting slightly more (10.54%) than high-traffic sites (8.27%), suggesting a long tail of smaller sites following SEO-blog advice rather than a top-down enterprise push.
So: roughly 1 in 10 sites now has the file. The question that matters is whether it does anything once it's there.
The Effectiveness Data: It Doesn't
This is where the 2026 evidence gets consistent, and consistently negative.
AI crawlers barely request it. Ahrefs analysed 137,210 domains in its Web Analytics panel and found that 97% of the llms.txt files it saw received zero requests in May 2026 — of roughly 38,000 valid files, only about 1,100 got any traffic at all. A separate 90-day server-log experiment by OtterlyAI found that of 62,100 AI bot requests logged, only 84 — about 0.1% — targeted llms.txt, while an average page on the same site drew around 265 AI bot visits in the same window. At a larger scale again, Limy analysed 515,382,577 AI bot traffic events over 90 days and counted just 408 that hit an llms.txt file — a rounding error against total AI crawler traffic.
No correlation with citations. SE Ranking's 300,000-domain study ran both statistical correlation tests and an XGBoost machine-learning model to see whether llms.txt presence predicted how often a domain gets cited by AI answers. The result: removing the llms.txt feature from the model improved its prediction accuracy — the file was adding noise, not signal. A separate analysis by ALLMO.ai, looking at over 94,000 AI citations, found llms.txt referenced in fewer than 0.002% of them.
No official support from the model makers. We could not find published documentation from OpenAI, Anthropic, or Google stating that llms.txt is used to decide what to crawl, index, or cite. Their public crawler guidance covers user agents and robots.txt, and llms.txt doesn't appear in it. Google has been the most explicit: its Search Advocates have called the file "purely speculative" for now and said Search doesn't use it. Absence of documentation isn't proof a crawler ignores the file — but it does mean nobody has committed to reading it.
Google has said so explicitly. Google's AI-optimization guidance, updated June 15, 2026, states: "You don't need to create new machine readable files, AI text files, markup, or Markdown to appear in Google Search (including its generative AI capabilities), as Google Search itself doesn't use them." Google's John Mueller has repeatedly compared llms.txt to the meta keywords tag — a self-declared, unverifiable file that search engines learned to ignore over a decade ago precisely because site owners control it and have every incentive to game it.
Put together: adoption is real, usage by the systems it targets is not.
Where It Actually Helps
There's one place the data doesn't contradict the pitch: coding assistants and docs retrieval. Tools like Cursor, Claude Code, and Windsurf — along with docs platforms like Mintlify — use structured files (often via an MCP server rather than a blind crawl) to fetch the right page from a documentation site with less token waste than parsing full HTML. This is a narrower claim than "gets you cited in AI search," and it holds up because it doesn't depend on a third-party crawler choosing to fetch the file unprompted — the assistant is explicitly pointed at the docs site by a developer who wants that specific content.
That's a real, if modest, engineering-productivity win. It's not an SEO or AI-visibility strategy, and treating it as one in 2026 isn't supported by the data.
Should You Bother?
| Scenario | Verdict |
|---|---|
| You maintain public developer docs and want coding assistants to retrieve them cleanly | Worth doing — low effort, plausible benefit |
| You're doing it hoping it improves ChatGPT/Perplexity/AI Overview citations | Not supported by current data — don't expect a lift |
| You're doing it instead of writing genuinely citable, well-structured content | Wasted effort — the content itself is what gets cited, not a metadata file describing it |
| Your platform (e.g., Mintlify) generates it automatically at no cost | Leave it on — costs nothing, might help the narrow docs-retrieval case |
The file costs almost nothing to maintain, so there's little harm in publishing one if your docs platform generates it for free. The mistake is treating it as a checkbox for AI search visibility, or worse, letting it substitute for the actual work — clear page structure, direct answers near the top of content, accurate schema markup — that the citation studies above show LLMs actually draw on.
If you're evaluating where to invest engineering time on AI-visibility work more broadly — structured data, crawl access, content architecture — that's a scoping question worth getting right before writing code; see how we approach that kind of technical audit on our About page.
FAQ
Does llms.txt improve my ranking in ChatGPT or Google AI Overviews?
No measurable evidence supports this as of mid-2026. Google states Search ignores the file entirely, and independent studies covering hundreds of thousands of domains found no statistical correlation between having an llms.txt and AI citation frequency.
Do AI crawlers even request llms.txt files?
Rarely. Multiple 90-day crawl-log studies in 2026 found llms.txt requests made up roughly 0.1% or less of total AI bot traffic, and the majority of domains with a valid file received zero requests for it in a given month.
Is llms.txt the same as robots.txt?
No. robots.txt is a decades-old, widely respected standard that controls crawler access (allow/disallow). llms.txt is a 2024 proposal, not an IETF or W3C standard, that attempts to curate content for LLMs — and unlike robots.txt, there's no confirmed mechanism by which major AI providers actually consume it.
Should I remove llms.txt if I already have one?
No need. There's no evidence it hurts anything, and if your docs platform (like Mintlify) generates it automatically, it may still help AI coding assistants fetch your documentation more efficiently. Just don't treat it as an AI-SEO lever.
Sources
- Rankability: LLMS.txt Adoption — 8.7% of the Top 1,000 (June 2026)
- Search Engine Journal: LLMs.txt Shows No Clear Effect On AI Citations, Based On 300k Domains
- Search Engine Journal: Google Says LLMs.txt Is "Purely Speculative" For Now
- Ahrefs: We Analyzed 137K Sites — 97% of llms.txt Files Never Get Read
- OtterlyAI: llms.txt and AI Visibility — Results from OtterlyAI's GEO Study
- Limy: LLMs.txt in 2026 — The Full Guide (515M AI bot events analysed)
- Kai Spriestersbach (Medium): The llms.txt is dead. More precisely: a dud.
- Search Engine Land: Meet llms.txt, a proposed standard for AI website content crawling
- Mintlify: The value of llms.txt — Hype or real?
- Mintlify: llms.txt documentation