What Is llms.txt, and Does Your Website Actually Need One?

Here’s the honest answer: llms.txt is a proposed standard for telling AI systems what your website contains, and for most Canadian businesses it isn’t worth doing yet. Google has said it doesn’t use the file for Search or AI Overviews. Independent tracking suggests almost no AI crawler requests it. If you run developer documentation, the calculation changes. If you run a brokerage, a clinic, or a storage facility, it doesn’t.

That’s a less exciting answer than the tools selling you a generator would like. It’s also the one that saves you an afternoon.

What Is llms.txt?

llms.txt is a plain text file in Markdown format that sits at the root of your website, at yoursite.com/llms.txt. It gives large language models a curated summary of your site: what you do, which pages matter, and where to find them.

Jeremy Howard proposed it in September 2024. The reasoning was sound. AI models have limited context windows, and a modern web page is mostly navigation, scripts, ads, and markup wrapped around a small amount of actual content. Rather than making a model parse all that, llms.txt hands it a clean map.

It’s worth being precise about what it is not. It isn’t an official standard, it isn’t ratified by any of the AI companies, and it doesn’t control anything. It’s a proposal that some sites have adopted voluntarily.

What Problem Is It Trying to Solve?

The problem is real even if this particular solution hasn’t landed.

AI systems increasingly decide what your business is based on what they can read quickly. If your site buries the answer to “what does this company do” three clicks deep inside a JavaScript-rendered page, a model may summarize you incorrectly or skip you entirely. That’s a genuine visibility risk, and it’s the same risk that answer engine optimization exists to address.

llms.txt proposes to solve it with a file. The alternative, which is what actually works today, is to solve it with the pages themselves.

What Does an llms.txt File Look Like?

It’s Markdown, and it’s short. A basic version looks like this:

# Company Name

> One-sentence description of what the business does.

## Core pages
– [Services](https://example.com/services/): What we offer and to whom
– [About](https://example.com/about/): Team, credentials, history

## Resources
– [Blog](https://example.com/blog/): Guides and industry analysis

That’s the whole format. An H1 for the name, a blockquote summary, then linked sections with a short note on each. Some sites also publish llms-full.txt, which contains the full text of the listed pages rather than just links.

Does Google Use llms.txt?

No, and Google has been unusually direct about it. Its 2026 guidance on optimizing for AI features states that llms.txt is not needed for AI Overviews, AI Mode, or any other generative feature in Search. Google Search Advocate John Mueller has said plainly that “no AI system currently uses llms.txt.”

There’s a wrinkle worth knowing, because it confuses people. Google did add an llms.txt check to Chrome’s Lighthouse tool, filed under Discoverability in a new agentic browsing category. That sounds like an endorsement until you read what the audit does. It only flags a server error when something goes wrong fetching the file. If you have no llms.txt at all, the audit returns Not Applicable. You are not penalized for skipping it.

So the two positions aren’t actually in conflict. Google’s tooling acknowledges the convention exists. Google’s Search guidance says it changes nothing for you. Both are true.

Who’s Actually Publishing One?

Look at the list of adopters and a pattern jumps out immediately. Stripe, Cloudflare, Anthropic, Hugging Face, Vercel, Zapier, and Cal.com all publish an llms.txt file. Every one of them is a developer-facing company whose primary content is technical documentation.

That isn’t a coincidence, and it’s the most useful signal in this whole debate. These companies aren’t adding the file hoping for a ranking bump. They’re adding it because their users genuinely point AI coding assistants at their docs, and a clean index makes those assistants give better answers about their product. The file solves a real problem for them.

Notice who isn’t on that list: local service businesses, professional practices, retailers, and B2B companies selling to non-technical buyers. Not because they haven’t caught up, but because the use case doesn’t exist for them. If nobody is asking Cursor how to integrate with your storage facility, an llms.txt file has no audience.

The useful question isn’t “have big companies adopted this.” It’s “have companies like mine adopted this, and for what reason.” For most Canadian small and mid-sized businesses, the honest answer to the second half is that there isn’t one yet.

Is llms.txt Worth It?

For most businesses, no. The clearest evidence is adoption on the crawler side rather than the publisher side. A 90-day study by OtterlyAI found that of 62,100 AI bot requests, only 84 hit the llms.txt file. That’s roughly one in a thousand.

Here’s the split that actually matters:

 
The cost of adding one is genuinely low, which is why “just do it, it can’t hurt” is common advice. The honest counterpoint is that a stale llms.txt is worse than none. It’s a file you now have to maintain, and one that misdescribes your site is a liability rather than an asset.

How to Create an llms.txt File

If you’ve read the table above and you’re in the yes column, here’s the process.

1. Pick the pages that define you. Ten to twenty is plenty. Core service pages, your about page, your best reference content. Not every URL on the site.
2. Write the file in Markdown. Follow the structure shown earlier. One line describing each link, written for a machine that has no other context about your business.
3. Upload it to your root directory so it resolves at yoursite.com/llms.txt. Not in a subfolder.
4. Confirm it loads by visiting the URL directly and checking it returns plain text, not a 404 or your site’s error page.
5. Put a reminder in your calendar. If the file still lists services you dropped a year ago, it’s actively working against you.

What to Do Instead

If your goal is for AI systems to find, understand, and cite your business, the work that moves the needle is on your pages, not in a text file.

Answer questions directly in the first sentence of a section so a model can extract a clean response. Use structured data so systems know what your content is without guessing. Build consistent mentions of your brand alongside your topic across the web. Keep your best pages current, because AI systems regenerate answers constantly and favour fresh sources. Our LLM-friendly content checklist covers the formatting side, and our 2026 SEO trends analysis covers where AI search is heading.

None of that is as quick as uploading a file. All of it is what’s actually working, and it’s the substance of our LLM SEO and generative engine optimization work.

Frequently Asked Questions

Does llms.txt replace robots.txt?

No. They do unrelated jobs. Robots.txt tells crawlers what they may and may not access, and it’s a long-established standard that every major crawler respects. llms.txt is a proposed summary of your content meant to help models understand it. Robots.txt restricts, llms.txt describes.

Does llms.txt help SEO?

No. Google’s 2026 guidance is explicit that the file isn’t needed for AI Overviews, AI Mode, or any generative Search feature, and it has no bearing on rankings. Any tool promising ranking improvements from an llms.txt file is overselling.

Do AI systems actually read llms.txt?

Rarely. OtterlyAI’s 90-day study found that 84 of 62,100 AI bot requests hit the file, about 0.1%. Some coding assistants and a few AI products do retrieve it, which is why developer documentation is the one clear use case, but general-purpose crawlers mostly ignore it.

Should I create an llms.txt file?

Only if you publish developer documentation or an API reference, where AI coding tools genuinely use it. For a typical local or service business, the answer today is no. Revisit if a major AI provider announces official support.

Where does the llms.txt file go?

In your website’s root directory, so it loads at yoursite.com/llms.txt. It has to be reachable at that exact path to be found, the same way robots.txt works. A file in a subfolder won’t be picked up.

What’s the difference between llms.txt and llms-full.txt?

llms.txt is a short index of your key pages with a line describing each. llms-full.txt contains the actual full text of those pages in one file. The full version is much larger and is mainly used by documentation sites that want a model to ingest everything at once.

llms.txt is worth understanding and, for most businesses, worth skipping this year. The visibility problem it points at is real, but the answer is better pages rather than a better file. Keep an eye on it, because that could change quickly if one of the major AI providers announces support, and a proposal with real backing would be a different conversation. Until then, the hours are better spent making the content you already have easier for both people and machines to use. Ready to turn your marketing into a growth engine? Claim your free 30-minute strategy session with Consultus Digital or call 416-460-1810.