Key facts
- The format was proposed by Jeremy Howard of Answer.AI in September 2024.
- It is a community proposal, not an official standard, and AI companies have not committed to reading it.
- It complements robots.txt and sitemap.xml rather than replacing them.
- Some sites also publish llms-full.txt with the full text of their key pages.
What goes in an llms.txt file
- 1A top level heading with the name of the site or product.
- 2A short summary in a blockquote: what it is and who it is for.
- 3Optional notes with important facts.
- 4Sections of links, each with a one line description of the page.
- 5An Optional section for pages that can be skipped when space is short.
Should you add one?
It takes minutes and has no downside, so it is worth adding. Treat it as a helpful map rather than a ranking factor: what still matters most is that AI crawlers can reach your pages and that those pages answer questions clearly. Keep it short, current and pointed at your best pages.
Put this into practice with Variask Site health and the free llms.txt generator. See Variask pricing or read the Variask FAQ.