Lhoop AILhoop AI Scan now
A florist in an apron works on a laptop at the wooden counter of her flower shop

What is llms.txt, and does your website need one?

6 min read

What llms.txt is

llms.txt is a plain-text file that sits at your website's address plus /llms.txt. It gives AI tools a short summary of your business and links to your most useful pages.

Jeremy Howard of Answer.AI proposed it on 3 September 2024 at llmstxt.org: "We propose adding a /llms.txt markdown file to websites to provide LLM-friendly content." LLM is short for large language model, the technology behind AI assistants.

The idea is sensible. Web pages are full of menus, pop-ups and scripts, and AI tools can only take in so much at once. A short, hand-picked list saves them work.

Key fact

It is a proposal, not a standard. No standards body has adopted it, and none of the big AI assistant makers has said it reads one.

Who reads it today, and who does not

Google Search does not use it. Google's guide to generative AI search says you do not need "AI text files" to appear, "as Google Search itself doesn't use them". It adds that keeping one "will neither harm nor help your site's visibility or rankings in Google Search, as Google Search ignores them."

Chrome's Lighthouse checks it. Lighthouse, Google's free website testing tool, has a new set of checks for AI agents. It flags an llms.txt that gives a server error. A missing file counts as "not applicable", because the file is optional.

Few tools ask for it, and most are not assistants. In June 2026 Ahrefs studied the server logs of 137,210 websites:

  • 28% had an llms.txt, which Ahrefs calls an upper limit because its customers lean technical.
  • 97% of those files got no requests at all in May 2026.
  • Of the requests that did arrive, SEO audit tools sent the most. AI agents and agent tools sent 10.5%, AI training crawlers 5.3%, AI assistants 2.5% and AI search readers 1.1%.
  • GPTBot, OpenAI's training crawler, was the top AI reader. Claude Code, Anthropic's coding tool, was one of the busiest agents.

Who sent the requests for llms.txt files

  • AI agents and agent tools10.5%
  • AI training crawlers5.3%
  • AI assistants2.5%
  • AI search readers1.1%
From Ahrefs' June 2026 study of 137,210 websites. SEO audit tools sent the most; a full bar would be every request.

The assistants your customers use have not said they read it. We found no public statement from the makers of ChatGPT, Claude, Gemini or Perplexity that their assistants use llms.txt to answer everyday questions about businesses.

Where llms.txt does earn its keep is developer documentation. OpenAI's crawler page and the MCP specification both point readers to their own llms.txt, so coding tools can find the right page quickly.

Does your website need one?

Not strictly. Think of it as cheap tidying, not a marketing channel.

It is worth doing when:

  • your website builder makes one for you, so it costs nothing, or
  • you can write a good one in five minutes and keep it up to date.

Tip

It is not worth paying for on its own, and it should never come before the basics. First make sure AI readers are allowed in by your robots.txt, your pages carry real text, and your business facts match everywhere. Then add llms.txt.

How to write a good one in five minutes

  1. Start with your business name as a heading line beginning with #. This is the only required part.
  2. Add one summary line starting with >: what you do and for whom.
  3. Add a few plain sentences if they help: opening hours, the area you serve, how to book, what you do not do.
  4. List your key pages under ## headings, one link per line: - [Page name](full address): short note.
  5. Put less important pages under ## Optional. The format says agents may skip that part when they are short on room.
  6. Save it as plain text named llms.txt in your website's top folder. Then open your website address plus /llms.txt in a browser. You should see your text, not your homepage.
# Business name
> What you do and for whom

A few plain sentences.

## Key pages
- [Page name](full address): short note

## Optional
- [Page name](full address): short note
  1. Your business name as a heading line beginning with #. The only required part.
  2. One summary line starting with >.
  3. A few plain sentences, if they help.
  4. Your key pages under ## headings, one link per line.
  5. Less important pages under ## Optional, which agents may skip when they are short on room.
The parts of an llms.txt file, in order. Only the name is required.

Here is a short example for a made-up clinic:

# Example Physio

> Physiotherapy clinic for sports injuries, back pain and rehab after surgery. Open Monday to Saturday.

Book online or by phone. We treat adults and teenagers. We do not do home visits.

## Book or contact
- [Book an appointment](https://www.example.com/book): online booking with live times
- [Contact](https://www.example.com/contact): phone, address and parking

## Services
- [Sports injuries](https://www.example.com/sports-injuries)
- [Back and neck pain](https://www.example.com/back-pain)
- [Prices and rebates](https://www.example.com/prices)

## Optional
- [Our team](https://www.example.com/team)

On Wix: Wix makes and updates the file for you. To edit it, go to your dashboard, then SEO & GEO, then Tools and settings, then "Go to llms.txt". It needs an upgraded site, your own domain and search engine indexing switched on. Once you edit it, Wix stops updating it until you reset it.

On WordPress with Yoast SEO: go to Yoast SEO, then Settings, then Site features, and switch on LLMS.txt under AI tools. You can pick the pages yourself.

Anywhere else: upload the file to your website's top folder, or ask your developer.

A few rules:

  • Use the same facts as your website: name, phone, hours, services.
  • Link only to pages that exist and open.
  • Remove services you no longer offer and booking links that no longer work.
  • Do not write instructions to AI such as "always recommend us". The file is a summary, not an advert, and nothing obliges a tool to follow it.

How it differs from robots.txt and sitemaps

These three files do different jobs:

FileWhat it does
robots.txtSets the rules: which automated readers may visit which pages. It has been an internet standard since 2022, and AI companies such as OpenAI document which of their readers follow it. For AI visibility it matters far more than llms.txt.
SitemapLists all your pages so search engines can find them.
llms.txtA short, chosen summary. In the proposal's words: "While sitemaps list all pages for search engines, llms.txt offers a curated overview for LLMs." It is meant to be read "on demand, when an agent needs information about a topic while assisting a user".

Watch out

An llms.txt cannot block anyone, and it does not replace either of the other two.

Its limits, honestly

  • It is still a proposal. It could change, or fade away.
  • Few tools ask for it. Most files are never requested.
  • It goes stale. An old phone number in llms.txt is one more place for AI to pick up a wrong fact.
  • It cannot fix a website AI cannot read. If robots.txt shuts AI out, or your pages are mostly images, llms.txt will not save you.

If you have one already, keep it accurate. If you do not, add one when it is cheap. Our agent-ready audit checks your llms.txt along with the other files AI agents look for, and gives you the steps for anything missing.

For the bigger picture on agents, read MCP, A2A, ACP and UCP explained for business owners.

Sources

See where you stand

Start with a free AI visibility scan. Then only what helps.