llms.txt: what it is, whether it works, and how to write one

What is llms.txt?

Language models have limited context, and web pages are full of navigation, scripts and ads. The llms.txt proposal offers a shortcut: one plain markdown file at the root of your site that says what the site is and points to the pages that matter, written for a model to read.

The proposal was published by Jeremy Howard on September 3, 2024, at llmstxt.org. It also suggests that sites offer clean markdown versions of individual pages at the same URL with .md added, such as page.html.md, so tools can read the content without the HTML around it.

Think of it as a curated table of contents for machines. It does not grant or deny access to anything. That remains the job of robots.txt.

Do AI engines actually use llms.txt?

Be clear-eyed here. llms.txt is a community proposal, not a standard from a standards body. As of this writing, none of the major AI providers has publicly confirmed that its crawlers or answer engines read llms.txt when deciding what to cite.

Google's guidance on AI features in Search says you do not need new machine-readable files or special markup to appear in AI Overviews or AI Mode. Normal SEO practice is what counts there.

Where llms.txt does help today is with tools that a person points at your site directly. Developer assistants, coding agents and documentation tools can fetch the file to get a quick, accurate map of your product. That is why many software documentation sites publish one.

So treat it as a low-cost addition. It takes an hour to write, has no known downside, and gives any tool that reads it a clean summary. Do not expect it to change your rankings or AI citations on its own.

What goes in an llms.txt file?

The proposal defines a simple structure in a fixed order. Only the first element is required.

• An H1 with the name of the project or site. This is the only required part.
• A blockquote with a short summary that contains the key facts needed to understand the rest of the file.
• Optional paragraphs or lists with more detail, but no extra headings.
• H2 sections that each hold a list of links, written as [name](url), optionally followed by a colon and a note about the page.
• An optional H2 section titled Optional, for secondary links that a tool can skip when it needs a shorter context.

How to write an llms.txt file

Below is the real llms.txt that seoamazing.com serves. It follows the format: a name, a one-paragraph summary in a blockquote, a short detail paragraph, and two link sections with notes.

1. Write the H1 with your product or site name exactly as you use it elsewhere.
2. Write a two or three sentence summary in a blockquote. Say what you do, for whom, and what makes it different, using plain facts.
3. Add a short paragraph with any details a model would need to describe you accurately, such as what you offer and who built it.
4. Group your most useful pages under H2 headings, such as Product, Docs or Company. Add a short note after each link.
5. Save the file as plain text at the root of your domain, so it loads at yourdomain.com/llms.txt.
6. Update it whenever pricing, product names or key pages change.

# SEOAmazing

> SEOAmazing is AI on-page SEO automation for any CMS. Paste one pixel, our crawler audits your site, AI writes the fixes (titles, meta descriptions, schema, headings, internal links), you approve them, and they go live from the edge in minutes. No CMS migration and no redeploys. Approved changes are also prerendered for AI crawlers that can't run JavaScript (GPTBot, ClaudeBot, PerplexityBot), so pages can be cited in AI answers.

SEOAmazing also tracks visibility where people search today: Google rank tracking (position, answer box and "People also ask" ownership), an optional read-only Google Search Console connection, and AI Visibility checks across ChatGPT, Claude, Gemini and Perplexity. Recon missions run browser agents for site audits, competitor content scans and lead lists. Built by The App Maker Pro.

## Product

- [How SEOAmazing works](https://seoamazing.com/instructions): step-by-step guide to the pixel, crawl, AI recommendations, rank tracking, Search Console and Recon missions
- [Pricing](https://seoamazing.com/pricing): Free, Starter ($39/mo), Pro ($99/mo), Agency ($299/mo); 30-day money-back guarantee
- [FAQ](https://seoamazing.com/faq): answers about the pixel, AI crawlers, rank tracking, billing and data privacy

## Company

- [About](https://seoamazing.com/about)
- [Contact](https://seoamazing.com/contact): ceo@theappmakerpro.com
- [Privacy policy](https://seoamazing.com/privacy)
- [Terms of service](https://seoamazing.com/terms)

What is llms-full.txt?

Some sites, mostly software documentation, also publish llms-full.txt. Instead of links, it contains the full text of the documentation in one markdown file, so a tool can load everything in a single request.

llms-full.txt is not part of the original llmstxt.org proposal. It is a convention that grew alongside it. It suits documentation that changes often and that developers feed into coding tools. For a typical business or marketing site, a short llms.txt is enough.

Common llms.txt mistakes to avoid

• Treating it as access control. llms.txt does not block or allow any crawler. Use robots.txt for that.
• Writing marketing copy. Superlatives and slogans give a model nothing useful. State facts.
• Letting it go stale. Old prices or retired product names can spread into AI descriptions of you.
• Linking every page. It is a curated list. Link the pages that best explain what you do, and leave the rest to your sitemap.
• Contradicting your site. The summary should match what your pages say, word for word on key facts.
• Serving it as HTML. It should be a plain text markdown file, not a web page.
• Expecting ranking gains. No search engine or AI provider has said it uses llms.txt as a ranking or citation signal.

llms.txt vs robots.txt vs sitemap.xml

These three files do different jobs, and llms.txt replaces neither of the others.

robots.txt tells crawlers what they may and may not fetch. It is where you allow or block bots such as GPTBot, ClaudeBot, PerplexityBot or OAI-SearchBot, and the major AI companies document that their crawlers follow it. A sitemap lists the URLs you want crawled and indexed. llms.txt is a short, human-written guide to what your site is and where the best information lives.

Get robots.txt and your sitemap right first, since crawlers rely on them. Then add llms.txt as a small extra. SEOAmazing's crawler starts from your sitemap when you add a site, which is one more reason to keep it accurate.

Frequently asked questions