{"id":50129,"date":"2026-08-10T12:16:18","date_gmt":"2026-08-10T12:16:18","guid":{"rendered":"http:\/\/consultusdigital.com\/blog\/what-is-llms-txt\/"},"modified":"2026-08-20T07:40:32","modified_gmt":"2026-08-20T07:40:32","slug":"what-is-llms-txt","status":"publish","type":"post","link":"https:\/\/consultusdigital.com\/blog\/what-is-llms-txt\/","title":{"rendered":"What Is llms.txt, and Does Your Website Actually Need One?"},"content":{"rendered":"

Here\u2019s the honest answer: llms.txt is a proposed standard for telling AI systems what your website contains, and for most Canadian businesses it isn\u2019t worth doing yet. Google has said it doesn\u2019t use the file for Search or AI Overviews. Independent tracking suggests almost no AI crawler requests it. If you run developer documentation, the calculation changes. If you run a brokerage, a clinic, or a storage facility, it doesn\u2019t.<\/span><\/p>\n

That\u2019s a less exciting answer than the tools selling you a generator would like. It\u2019s also the one that saves you an afternoon.<\/span><\/p>\n

What Is llms.txt?<\/b><\/h2>\n

llms.txt is a plain text file in Markdown format that sits at the root of your website, at yoursite.com\/llms.txt. It gives large language models a curated summary of your site: what you do, which pages matter, and where to find them.<\/span><\/p>\n

Jeremy Howard proposed it in September 2024. The reasoning was sound. AI models have limited context windows, and a modern web page is mostly navigation, scripts, ads, and markup wrapped around a small amount of actual content. Rather than making a model parse all that, llms.txt hands it a clean map.<\/span><\/p>\n

It\u2019s worth being precise about what it is not. It isn\u2019t an official standard, it isn\u2019t ratified by any of the AI companies, and it doesn\u2019t control anything. It\u2019s a proposal that some sites have adopted voluntarily.<\/span><\/p>\n

What Problem Is It Trying to Solve?<\/b><\/h2>\n

The problem is real even if this particular solution hasn\u2019t landed.<\/span><\/p>\n

AI systems increasingly decide what your business is based on what they can read quickly. If your site buries the answer to \u201cwhat does this company do\u201d three clicks deep inside a JavaScript-rendered page, a model may summarize you incorrectly or skip you entirely. That\u2019s a genuine visibility risk, and it\u2019s the same risk that <\/span>answer engine optimization<\/a><\/strong> exists to address.<\/span><\/p>\n

llms.txt proposes to solve it with a file. The alternative, which is what actually works today, is to solve it with the pages themselves.<\/span><\/p>\n

What Does an llms.txt File Look Like?<\/b><\/h2>\n

It\u2019s Markdown, and it\u2019s short. A basic version looks like this:<\/span><\/p>\n

# Company Name<\/span>
<\/span>
<\/span>> One-sentence description of what the business does.<\/span>
<\/span>
<\/span>## Core pages<\/span>
<\/span>\u2013 [Services](https:\/\/example.com\/services\/): What we offer and to whom<\/span>
<\/span>\u2013 [About](https:\/\/example.com\/about\/): Team, credentials, history<\/span>
<\/span>
<\/span>## Resources<\/span>
<\/span>\u2013 [Blog](https:\/\/example.com\/blog\/): Guides and industry analysis<\/span><\/p>\n

That\u2019s the whole format. An H1 for the name, a blockquote summary, then linked sections with a short note on each. Some sites also publish llms-full.txt, which contains the full text of the listed pages rather than just links.<\/span><\/p>\n

Does Google Use llms.txt?<\/b><\/h2>\n

No, and Google has been unusually direct about it. Its 2026 guidance on optimizing for AI features states that llms.txt is not needed for AI Overviews, AI Mode, or any other generative feature in Search. Google Search Advocate John Mueller has said plainly that \u201cno AI system currently uses llms.txt.\u201d<\/span><\/p>\n

There\u2019s a wrinkle worth knowing, because it confuses people. Google did add an llms.txt check to Chrome\u2019s Lighthouse tool, filed under Discoverability in a new agentic browsing category. That sounds like an endorsement until you read what the audit does. It only flags a server error when something goes wrong fetching the file. If you have no llms.txt at all, the audit returns Not Applicable. You are not penalized for skipping it.<\/span><\/p>\n

So the two positions aren\u2019t actually in conflict. Google\u2019s tooling acknowledges the convention exists. Google\u2019s Search guidance says it changes nothing for you. Both are true.<\/span><\/p>\n

Who\u2019s Actually Publishing One?<\/b><\/h2>\n

Look at the list of adopters and a pattern jumps out immediately. Stripe, Cloudflare, Anthropic, Hugging Face, Vercel, Zapier, and Cal.com all publish an llms.txt file. Every one of them is a developer-facing company whose primary content is technical documentation.<\/span><\/p>\n

That isn\u2019t a coincidence, and it\u2019s the most useful signal in this whole debate. These companies aren\u2019t adding the file hoping for a ranking bump. They\u2019re adding it because their users genuinely point AI coding assistants at their docs, and a clean index makes those assistants give better answers about their product. The file solves a real problem for them.<\/span><\/p>\n

Notice who isn\u2019t on that list: local service businesses, professional practices, retailers, and B2B companies selling to non-technical buyers. Not because they haven\u2019t caught up, but because the use case doesn\u2019t exist for them. If nobody is asking Cursor how to integrate with your storage facility, an llms.txt file has no audience.<\/span><\/p>\n

The useful question isn\u2019t \u201chave big companies adopted this.\u201d It\u2019s \u201chave companies like mine adopted this, and for what reason.\u201d For most Canadian small and mid-sized businesses, the honest answer to the second half is that there isn\u2019t one yet.<\/span><\/p>\n

Is llms.txt Worth It?<\/b><\/h2>\n

For most businesses, no. The clearest evidence is adoption on the crawler side rather than the publisher side. A 90-day study by OtterlyAI found that of 62,100 AI bot requests, only 84 hit the llms.txt file. That\u2019s roughly one in a thousand.<\/span><\/p>\n

Here\u2019s the split that actually matters:<\/span><\/p>\n\n\n\n\n\n\n\n\n
Your situation<\/th>\nWorth doing?<\/th>\nWhy<\/th>\n<\/tr>\n<\/thead>\n
Developer docs or API reference<\/td>\nYes<\/td>\nCoding assistants like Cursor and GitHub Copilot do read these files. Stripe, Cloudflare, and Anthropic all publish one.<\/td>\n<\/tr>\n
SaaS product with technical users<\/td>\nMaybe<\/td>\nLow cost, plausible upside if your audience uses AI coding tools.<\/td>\n<\/tr>\n
Local service business, clinic, brokerage, retailer<\/td>\nNot yet<\/td>\nYour customers aren\u2019t reaching you through coding agents, and Google confirms it does nothing for Search.<\/td>\n<\/tr>\n
Anyone hoping it lifts rankings<\/td>\nNo<\/td>\nIt doesn\u2019t, and no credible source claims otherwise.<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

\u00a0
The cost of adding one is genuinely low, which is why \u201cjust do it, it can\u2019t hurt\u201d is common advice. The honest counterpoint is that a stale llms.txt is worse than none. It\u2019s a file you now have to maintain, and one that misdescribes your site is a liability rather than an asset.<\/p>\n

How to Create an llms.txt File<\/b><\/h2>\n

If you\u2019ve read the table above and you\u2019re in the yes column, here\u2019s the process.<\/span><\/p>\n

    \n
  1. Pick the pages that define you.<\/b> Ten to twenty is plenty. Core service pages, your about page, your best reference content. Not every URL on the site.<\/span><\/li>\n
  2. Write the file in Markdown.<\/b> Follow the structure shown earlier. One line describing each link, written for a machine that has no other context about your business.<\/span><\/li>\n
  3. Upload it to your root directory<\/b> so it resolves at yoursite.com\/llms.txt. Not in a subfolder.<\/span><\/li>\n
  4. Confirm it loads<\/b> by visiting the URL directly and checking it returns plain text, not a 404 or your site\u2019s error page.<\/span><\/li>\n
  5. Put a reminder in your calendar.<\/b> If the file still lists services you dropped a year ago, it\u2019s actively working against you.<\/span><\/li>\n<\/ol>\n

    What to Do Instead<\/b><\/h2>\n

    If your goal is for AI systems to find, understand, and cite your business, the work that moves the needle is on your pages, not in a text file.<\/span><\/p>\n

    Answer questions directly in the first sentence of a section so a model can extract a clean response. Use structured data so systems know what your content is without guessing. Build consistent mentions of your brand alongside your topic across the web. Keep your best pages current, because AI systems regenerate answers constantly and favour fresh sources. Our <\/span>LLM-friendly content checklist<\/a><\/strong> covers the formatting side, and our <\/span>2026 SEO trends analysis<\/a><\/strong> covers where AI search is heading.<\/span><\/p>\n

    None of that is as quick as uploading a file. All of it is what\u2019s actually working, and it\u2019s the substance of our <\/span>LLM SEO<\/a><\/strong> and <\/span>generative engine optimization<\/a><\/strong> work.<\/span><\/p>\n

    Frequently Asked Questions<\/b><\/h2>\n

    Does llms.txt replace robots.txt?<\/b><\/h3>\n

    No. They do unrelated jobs. Robots.txt tells crawlers what they may and may not access, and it\u2019s a long-established standard that every major crawler respects. llms.txt is a proposed summary of your content meant to help models understand it. Robots.txt restricts, llms.txt describes.<\/span><\/p>\n

    Does llms.txt help SEO?<\/b><\/h3>\n

    No. Google\u2019s 2026 guidance is explicit that the file isn\u2019t needed for AI Overviews, AI Mode, or any generative Search feature, and it has no bearing on rankings. Any tool promising ranking improvements from an llms.txt file is overselling.<\/span><\/p>\n

    Do AI systems actually read llms.txt?<\/b><\/h3>\n

    Rarely. OtterlyAI\u2019s 90-day study found that 84 of 62,100 AI bot requests hit the file, about 0.1%. Some coding assistants and a few AI products do retrieve it, which is why developer documentation is the one clear use case, but general-purpose crawlers mostly ignore it.<\/span><\/p>\n

    Should I create an llms.txt file?<\/b><\/h3>\n

    Only if you publish developer documentation or an API reference, where AI coding tools genuinely use it. For a typical local or service business, the answer today is no. Revisit if a major AI provider announces official support.<\/span><\/p>\n

    Where does the llms.txt file go?<\/b><\/h3>\n

    In your website\u2019s root directory, so it loads at yoursite.com\/llms.txt. It has to be reachable at that exact path to be found, the same way robots.txt works. A file in a subfolder won\u2019t be picked up.<\/span><\/p>\n

    What\u2019s the difference between llms.txt and llms-full.txt?<\/b><\/h3>\n

    llms.txt is a short index of your key pages with a line describing each. llms-full.txt contains the actual full text of those pages in one file. The full version is much larger and is mainly used by documentation sites that want a model to ingest everything at once.<\/span><\/p>\n

    llms.txt is worth understanding and, for most businesses, worth skipping this year. The visibility problem it points at is real, but the answer is better pages rather than a better file. Keep an eye on it, because that could change quickly if one of the major AI providers announces support, and a proposal with real backing would be a different conversation. Until then, the hours are better spent making the content you already have easier for both people and machines to use. Ready to turn your marketing into a growth engine? Claim your free 30-minute strategy session with <\/span>Consultus Digital<\/a><\/strong> or call 416-460-1810<\/a>.<\/span><\/p>\n","protected":false},"excerpt":{"rendered":"

    Here\u2019s the honest answer: llms.txt is a proposed standard for telling AI systems what your website contains, and for most Canadian businesses it isn\u2019t worth doing yet. Google has said it doesn\u2019t use the file for Search or AI Overviews. Independent tracking suggests almost no AI crawler requests it. If you run developer documentation, the […]<\/p>\n","protected":false},"author":2,"featured_media":50130,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[16],"tags":[33,19,37],"class_list":["post-50129","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-seo","tag-digital-marketing","tag-digital-marketing-toronto","tag-seo"],"_links":{"self":[{"href":"https:\/\/consultusdigital.com\/blog\/wp-json\/wp\/v2\/posts\/50129","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/consultusdigital.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/consultusdigital.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/consultusdigital.com\/blog\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/consultusdigital.com\/blog\/wp-json\/wp\/v2\/comments?post=50129"}],"version-history":[{"count":0,"href":"https:\/\/consultusdigital.com\/blog\/wp-json\/wp\/v2\/posts\/50129\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/consultusdigital.com\/blog\/wp-json\/wp\/v2\/media\/50130"}],"wp:attachment":[{"href":"https:\/\/consultusdigital.com\/blog\/wp-json\/wp\/v2\/media?parent=50129"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/consultusdigital.com\/blog\/wp-json\/wp\/v2\/categories?post=50129"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/consultusdigital.com\/blog\/wp-json\/wp\/v2\/tags?post=50129"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}