llms.txt is a proposed convention for a markdown file placed at the root of a website (/llms.txt) that gives large language models a short, curated overview of the site: its name, a one-paragraph summary, and sections of links to the most important pages with brief descriptions. It is not an official standard, so treat it as a low-cost helper rather than a guaranteed AI ranking signal.
I added one to my own site as part of entity SEO work. Here is what it is, how to write it, and what to realistically expect.
Where does llms.txt come from?
Jeremy Howard of Answer.AI published the llms.txt proposal in September 2024. The idea: websites are built for humans and full of navigation, scripts and layout, while language models work best with concise, clean text. A small markdown file gives them a curated entry point, much like a table of contents.
What llms.txt is not
- Not an access control file. It does not block or allow crawling; that is robots.txt.
- Not a sitemap replacement. XML sitemaps list all URLs for search engine crawlers.
- Not a confirmed ranking signal. I have not seen any major AI search provider confirm that llms.txt affects whether you get cited. Google has said that no special files are needed to appear in its AI features.
So why bother? Because it takes little time, some AI coding tools and agents do read it, and writing it forces you to state clearly who you are and which pages matter.
The llms.txt format
The proposal defines a simple structure in markdown:
- An H1 with the name of the site or project (the only required part).
- A blockquote with a short summary.
- Optional paragraphs or lists with more detail.
- H2 sections containing lists of links, each with an optional description after a colon.
- An optional section titled “Optional” for secondary links that can be skipped when context is short.
llms.txt example
A realistic example for a freelance developer’s site:
# Jane Developer
> Jane Developer is a web developer building online stores and business
> websites on WooCommerce, Shopify and Laravel, with a focus on speed,
> technical SEO and conversion.
Jane works with clients in Europe and the Middle East, in English and Arabic.
## Services
- [Services overview](https://example.com/services/): What Jane builds and how projects work
- [Contact](https://example.com/contact/): How to start a project
## About
- [About Jane](https://example.com/about/): Background, skills and approach
- [Projects](https://example.com/projects/): Selected work with descriptions
## Guides
- [Core Web Vitals guide](https://example.com/core-web-vitals-guide/): LCP, INP and CLS explained
- [WooCommerce vs Shopify](https://example.com/woocommerce-vs-shopify/): How to choose a store platform
## Optional
- [Blog](https://example.com/blog/): All articles
Keep descriptions factual. This file may be read by a machine and quoted; exaggerations are a liability.
How to add llms.txt to WordPress
Option 1: a static file
Upload llms.txt to the WordPress root (the folder with wp-config.php). Simple, but you must update it manually.
Option 2: generate it dynamically
Serve it through WordPress so it stays in sync with your content:
add_action( 'init', function () {
add_rewrite_rule( '^llms.txt$', 'index.php?mx_llms=1', 'top' );
} );
add_filter( 'query_vars', function ( $vars ) {
$vars[] = 'mx_llms';
return $vars;
} );
add_action( 'template_redirect', function () {
if ( ! get_query_var( 'mx_llms' ) ) {
return;
}
header( 'Content-Type: text/plain; charset=utf-8' );
echo '# ' . get_bloginfo( 'name' ) . "nn";
echo '> ' . get_bloginfo( 'description' ) . "nn";
echo "## Guidesnn";
foreach ( get_posts( array( 'numberposts' => 20 ) ) as $p ) {
printf( "- [%s](%s)n", get_the_title( $p ), get_permalink( $p ) );
}
exit;
} );
Flush permalinks after adding a rewrite rule (Settings → Permalinks → Save, or wp rewrite flush). Some SEO plugins now offer llms.txt generation too.
Option 3: on other platforms
On hosted platforms where you cannot place files at the root, check whether the platform or an app supports it, or use a redirect if allowed.
Writing a good llms.txt
- Lead with a precise summary: who, what, for whom. Reuse your entity description.
- Link to canonical URLs only.
- Prefer pages with substantial, unique content.
- Group logically: services, about, guides, policies.
- Keep it short; the point is curation.
- Update it when key pages change.
Caching and serving
- Serve as
text/plainortext/markdownwith UTF-8. - Make sure caching plugins do not serve an HTML error page at that URL.
- Check it is not blocked by robots.txt or a firewall rule for bots you want to allow.
How to check it works
curl -sI https://example.com/llms.txt
curl -s https://example.com/llms.txt | head -20
Expect a 200 status and your markdown content.
Common mistakes
- Treating llms.txt as a magic AI ranking switch and neglecting real content.
- Listing hundreds of links, which defeats the curation.
- Including claims that are not on the site.
- Letting it go stale after a redesign.
The fundamentals still decide visibility: crawlable pages, clear facts and helpful content. See how to rank in AI Overviews. Want llms.txt and entity markup set up for your site? Contact me.
Frequently asked questions
What is llms.txt?
llms.txt is a proposed convention: a markdown file at /llms.txt that summarises a website and links to its key pages, so large language models and AI tools can understand it quickly at inference time.
Is llms.txt an official standard?
No. It is a community proposal published in 2024. It is not part of an official standards body, and major AI search providers have not committed to using it as a ranking or citation signal.
What is the difference between llms.txt and robots.txt?
robots.txt tells crawlers what they may access. llms.txt does not control access; it offers a curated summary and reading list for AI tools.
What is llms-full.txt?
Some sites also publish llms-full.txt, a single file with the full text of key documentation, so tools can load everything at once. It is a common extension, not a requirement.
