llms.txt Explained: Making Your Site Legible to AI

If you’ve watched assistants like ChatGPT, Claude, and Perplexity start answering questions your site used to answer, you’ve felt the shift. The web is being read by machines as often as people now, and most sites are hard for those machines to parse cleanly. llms.txt is a small proposal aimed squarely at that problem: a single Markdown file that hands large language models a curated, readable map of what your site is and where the important content lives.

It is not a ranking trick, and it is not magic. It’s a convention: early, optional, and still finding its footing. This is a practitioner’s read on what it is, how to write one well, how it relates to the files you already have, and where its real limits sit today.

What llms.txt is, and the proposal behind it

llms.txt is a plain-text Markdown file you place at the root of your domain: yourdomain.com/llms.txt. The proposal, introduced by Jeremy Howard of Answer.AI in 2024, starts from one observation: a language model working inside a limited context window can’t crawl your whole site, render your JavaScript, or wade through navigation, cookie banners, and ad markup to find the substance. So give it a clean index instead.

The format is deliberately simple. An H1 with your site or company name, an optional blockquote summary, then sections of Markdown links pointing to the pages that matter, each with a short description of what’s there. That’s the whole idea. It reads like a hand-built table of contents written for a reader who only has a few thousand tokens of attention to spend.

llms.txt isn’t a feed for crawlers to obey. It’s a brief for a model that’s already trying to understand you, written so it understands you correctly.

The distinction matters. robots.txt tells bots where they may not go. llms.txt tells a model where the good stuff is and what it means. One is a fence; the other is a tour guide.

How to write an llms.txt file

The spec is loose by design, but a good file follows a consistent shape. Here’s the structure, top to bottom.

1. An H1 with the name of the site or project. This is the only required line.
2. A blockquote summary. One or two sentences on what you do and who you serve. Models lean on this hard.
3. Optional context paragraphs. Plain prose explaining anything a model needs to interpret the links correctly, with no headings.
4. Sections of links. ## headings grouping related pages, each link followed by a colon and a short description.
5. An ## Optional section. Lower-priority links a model can skip when context is tight.

A minimal version for a boutique agency might read like this:

# Strynal

> Strynal is a boutique digital agency for branding and digital
> experiences, built for teams solving uncommon problems.

Strategy, brand, and build under one roof. Every engagement starts
on a blank page. No templates.

## Services

- [AI Visibility](https://strynal.com/services/ai-visibility): making
  brands legible and citable to AI assistants.
- [Websites & Apps](https://strynal.com/services/websites-apps): design
  and engineering for marketing sites and products.

## Writing

- [Generative Engine Optimization](https://strynal.com/blog/what-is-generative-engine-optimization):
  how to show up in AI answers.

## Optional

- [About](https://strynal.com/about): team and positioning.

A few principles separate a useful file from a box-ticking one.

• Curate, don’t dump. The point is signal. Link the pages that define you, not all 400 of them. If everything is important, nothing is.
• Write descriptions for a stranger. Each link’s gloss should make sense to a model with zero prior context about your brand. Say what the page is, not just what it’s called.
• Point at clean content. Links should resolve to pages a model can actually read: semantic HTML or Markdown, not a screen that needs JavaScript to paint its words. This is where machine-legibility and good structured data practice reinforce each other.
• Keep it current. A stale index is worse than none, because it confidently misdirects. Treat llms.txt as a living artifact, regenerated when your structure changes.

If your stack uses content collections (as ours does on Astro), generating llms.txt from the same source as your sitemap is straightforward, and it keeps the file honest by construction.

llms.txt vs llms-full.txt vs robots.txt and sitemap

These four files get conflated constantly. They do different jobs, and understanding the split is most of the battle.

llms.txt vs llms-full.txt

llms.txt is the index: links and descriptions, lightweight, meant to orient. llms-full.txt is the full text: the actual content of those pages concatenated into one large Markdown document, so a model can ingest your substance without following a single link.

The trade-off is context budget. llms.txt is small and cheap to load but requires the model to fetch pages it wants to read. llms-full.txt puts everything in one place but can run to hundreds of thousands of tokens, which overflows smaller context windows and bloats the prompt for larger ones. A reasonable pattern: ship llms.txt for everyone, and offer llms-full.txt for documentation-heavy sites where a model genuinely benefits from reading the whole corpus at once.

llms.txt vs robots.txt

robots.txt is a decades-old standard that governs access: which user agents may crawl which paths. It’s a permission boundary. llms.txt governs comprehension. It assumes a model is already reading and helps it read well. They don’t overlap, and they don’t replace each other. You still need robots.txt to manage crawler behavior, including the growing list of AI crawler user agents you may want to allow or block.

llms.txt vs sitemap.xml

A sitemap is a machine-oriented list of every URL you want indexed, with metadata like last-modified dates. It’s exhaustive and structural, built for search engine crawlers that will visit each entry. llms.txt is curated and semantic, built for a model that will read a handful of entries and needs to know which ones matter and why. A sitemap says “here is everything.” llms.txt says “here is what we are, and here’s where to start.”

Current limitations and realistic expectations

Here’s the part the breathless posts skip: as of early 2026, no major AI provider has publicly committed to consuming llms.txt as a ranking or retrieval signal. Google has said it isn’t using it. Adoption among model builders is uneven and largely unconfirmed. That doesn’t make the file useless, but it does change why you’d ship one.

Set expectations accordingly.

• It is not a guaranteed input. Treat llms.txt as a low-cost bet on a convention that may become standard, not as a lever with measurable returns today. If someone promises you traffic from it, be skeptical.
• It doesn’t make bad content good. A clean index pointing at thin, generic pages just helps a model find thin, generic pages faster. The work that actually earns AI citations is substance, clarity, and authority. That’s the throughline of how to get cited by AI assistants.
• It can leak intent. You’re publishing a prioritized map of what you consider important. That’s usually fine, occasionally not. Decide deliberately what belongs in it.
• It needs maintenance. An unmaintained llms.txt decays into misinformation about your own site. Only ship one you’ll keep current.

So why bother? Because the cost is near zero and the upside compounds with everything else you should already be doing for generative engine optimization. Writing a good llms.txt forces you to answer a clarifying question (what are the ten pages that actually define us?), and that exercise is worth the hour even if no model ever reads the file. When you also publish clean, semantic, fast-loading pages, you’re legible to machines whether or not this particular convention wins.

That’s the honest framing. llms.txt is a small, sensible, low-risk move that sits inside a much larger discipline. Don’t oversell it, and don’t skip it.

How Strynal approaches machine-legible sites

We treat AI legibility the way we treat performance and accessibility: something designed in from the first blank page, not bolted on after launch. A site that’s clear to a model is almost always clear to a person, because both reward semantic structure, honest content hierarchy, and pages that say what they are. The work that makes you legible to AI is mostly the same work that makes you good.

Because strategy, brand, and build sit under one roof here, the team that decides what your pages say is the team that ships the markup a model reads, so the index, the structured data, and the content never drift apart in a handoff nobody owns. That’s the spine of our AI visibility practice, and the standard we hold as the in-house studio for Global Digital Platforms.

If you’re weighing whether llms.txt is worth your time, or you want your site genuinely readable to the assistants your customers now ask, tell us what you’re building and we’ll help you find the moves that actually move the needle.