research

Is llms.txt Worth It? What the Data Says

July 19, 2026
5 min read
n=300,000
Dmitrij Żatuchin
llms.txtAI VisibilityGEOAEOTechnical SEOAI Crawlers

Short answer: llms.txt is worth about an hour, no more. The file is spreading fast, yet across the largest checks run so far it shows no measurable link to whether AI cites your brand. It does no direct harm. The real risk is that it feels like AI-visibility work and quietly replaces the work that moves citations. To see whether AI actually recommends your brand, run the free Answer Trail audit.

GitHub commits mentioning llms.txt grew about nine times between January and June 2026, while the file shows no link to being cited, p = 0.85

What is llms.txt?

llms.txt is a plain text file you place at the root of your website. It is meant to tell AI crawlers which pages matter and to hand them a clean version to read. The pitch is simple: it is the new robots.txt for AI. Add the file, the model reads it, and citations follow. That pitch is what this piece tests against data.

Is everyone really adding llms.txt?

Close to it, and the curve is steep. I pulled the GitHub Search API month by month: commits mentioning llms.txt grew from 5,986 in January 2026 to 52,692 in June, about nine times in six months. Pull requests roughly tripled. July ran hotter still, with 122,811 commits in the first nineteen days alone, already more than twice June's full month.

A jump that large in a single month has one likely cause: tooling. Documentation frameworks now emit llms.txt by default, so most of those commits are automated. Very few are a person deliberately deciding the file is worth it. The honest label for the curve is activity, not adoption. The deliberate-adoption number is smaller and calmer: about 5.6% of the top 10,000 sites carried the file by June 2026, up from roughly 1% a year earlier.

Does llms.txt help you get cited by AI?

This is the question that matters, and the largest checks so far say no. SE Ranking looked at around 300,000 domains and found that having the file "doesn't seem to directly impact AI citation frequency." A separate analysis by Trakkr, across tens of thousands of AI-cited domains, put the correlation at p = 0.85, a statistical way of saying no relationship at all. In one test, removing the file even improved the model's accuracy.

Back in early 2025, before the hype, adoption was near zero: about 15 sites across the Majestic Million (Chris Green's crawl, February 2025). So the file has earned a large audience of authors and almost no evidence of effect.

Do AI crawlers even read llms.txt?

By the accounts of the people who would know, mostly not. Google's John Mueller said it plainly: "you can tell when you look at your server logs that they don't even check for it." Ahrefs' own Ryan Law calls llms.txt "a solution in search of a problem." No major AI provider has confirmed using the file to build answers.

You do not have to take anyone's word for it. Open your server logs and search for the crawlers by name: GPTBot, ClaudeBot, Google-Extended, PerplexityBot. Look at what they request. If they never fetch /llms.txt, that settles it for your site in about five minutes.

So is llms.txt worth adding?

In the method behind the book Become the Answer, every claim gets one of three verdicts: evidence-backed, folklore, or honestly open. On the evidence so far, "add llms.txt and citations follow" is folklore.

The file does no direct harm, and that is exactly the trap. It costs an hour, it feels like progress, and it can quietly stand in for the work crawlers actually respond to. If your CMS makes it a two-minute job, fine, add it and move on. Do not build a project around it, and do not report it upward as AI-visibility work.

What should you do instead?

Two things move whether AI names your brand, and neither is a file at your root. First, what crawlers can reach and lift from your own pages: clean, answer-first content they can actually read. Second, and larger, where other sites cite you by name, because about 85% of the sources AI quotes when it answers a buyer question sit off your own website. The full method is in how to know if AI recommends your brand, and you can measure your own gaps free with Answer Trail.

Method

GitHub activity: GitHub Search API, commits and pull requests mentioning "llms.txt" by month, January to June 2026, pulled July 2026. This counts developer activity and is inflated by documentation tooling that auto-emits the file, so it is a measure of motion, not a census of deliberate adoption. Adoption share: Chris Green (Majestic Million crawl, February 2025), Casey Burridge and Rankability (2026). Citation link: SE Ranking (around 300,000 domains, November 2025) and Trakkr (Study 005, p = 0.85). Crawler behavior: John Mueller (r/TechSEO, 2025) and Ryan Law (Ahrefs).

Continue Reading

research

How to Know If AI Recommends Your Brand (and How to Fix It)

Ask AI the questions your buyers ask, see if your brand appears, and read the sources it cites. Free with Answer Trail, GDPR aligned. The fix: about 85% of what AI cites sits off your own website, so earn roughly five credible third-party mentions per buyer question to reach about 80% presence, then watch it over time because the cited sources turn over week to week.

article

The "Best Of" Trap: Why Self-Promotional Listicles Are Losing Google and Winning ChatGPT

Three signals in one week: Google penalized self-promotional listicles (SaaS brands losing 34-49% visibility), reduced crawl limits by 86.7%, while Ahrefs found 44% of ChatGPT citations come from those exact listicles. Google is shrinking, AI is expanding. Includes a 5-minute brand audit playbook and split strategy for navigating both discovery systems.

research

Where AI Gets Its Answers: 167,551 Citations Behind Brand Reputation

A cross-market citation dataset: 167,551 URL-grounded citations across 128 brands, 13 languages, and 12 markets from GPT, Gemini, and Perplexity. AI grounds brand answers in third-party sources 85.7% of the time; 18% of domains carry 80% of citations; Wikipedia is the top domain in 11 of 12 languages. Open under CC-BY-4.0 with DOI.

article

The State of AI Visibility: Why Brands Are Losing Control Inside AI Answers

Search didn't disappear. It stopped being the place where decisions happen. Traditional SEO assumed that if you ranked well, you would be discovered. AI doesn't work that way. This thought leadership piece defines why AI visibility matters and introduces the four dimensions of AI visibility: Presence, Relevance, Usefulness, and Trust.

BeVisible Club

Get research like this before we publish it.

New AI visibility studies, ranking patterns, and source-stack data delivered to your inbox a week before they go public.

Join BeVisible Club

Want to Know How AI Sees Your Brand?

Rankfor.AI measures your brand's AI visibility across all major platforms and provides actionable recommendations.

About the Author

Dmitrij Żatuchin

Founder & Researcher

Dmitrij Żatuchin is the founder of Rankfor.AI. A computer scientist with a PhD in semantic web technologies, he bridges the gap between how AI reasons about brands and how brands want to be understood. With over two decades of software architecture experience and academic roles at Estonian Business School and EUAS, Dmitrij builds the measurement infrastructure brands need to transition from optimizing for search engines to becoming visible for reasoning engines.

© 2025-2026 Rankfor.AI™ All rights reserved.

Ask AI about Rankfor.AI

Rankfor.AI sp. z o.o., Skarbowcow 23B, 53-025 Wroclaw, Poland

KRS: 0001190083 | NIP: 8993033605

We use cookies

We use essential cookies to make our site work. With your consent, we may also use analytics cookies to understand how you use our tools (like the Dice Roller) so we can improve them. Learn more