llms.txt is a proposed convention for a markdown file placed at the root of a website (/llms.txt) that gives large language models a short, curated overview of the site: its name, a one-paragraph summary, and sections of links to the most important pages with brief descriptions. It is not an official standard, so treat it as a low-cost helper rather than a guaranteed AI ranking signal.
I added one to my own site as part of entity SEO work. Here is what it is, how to write it, and what to realistically expect.
Where does llms.txt come from?
Jeremy Howard of Answer.AI published the llms.txt proposal in September 2024. The idea: websites are built for humans and full of navigation, scripts and layout, while language models work best with concise, clean text. A small markdown file gives them a curated entry point, much like a table of contents.
What llms.txt is not
- Not an access control file. It does not block or allow crawling; that is robots.txt.
- Not a sitemap replacement. XML sitemaps list all URLs for search engine crawlers.
- Not a confirmed ranking signal. I have not seen any major AI search provider confirm that llms.txt affects whether you get cited. Google has said that no special files are needed to appear in its AI features.
So why bother? Because it takes little time, some AI coding tools and agents do read it, and writing it forces you to state clearly who you are and which pages matter.
The llms.txt format
The proposal defines a simple structure in markdown:
- An H1 with the name of the site or project (the only required part).
- A blockquote with a short summary.
- Optional paragraphs or lists with more detail.
- H2 sections containing lists of links, each with an optional description after a colon.
- An optional section titled “Optional” for secondary links that can be skipped when context is short.
llms.txt example
A realistic example for a freelance developer’s site:
# Jane Developer
> Jane Developer is a web developer building online stores and business
> websites on WooCommerce, Shopify and Laravel, with a focus on speed,
> technical SEO and conversion.
Jane works with clients in Europe and the Middle East, in English and Arabic.
## Services
- [Services overview](https://example.com/services/): What Jane builds and how projects work
- [Contact](https://example.com/contact/): How to start a project
## About
- [About Jane](https://example.com/about/): Background, skills and approach
- [Projects](https://example.com/projects/): Selected work with descriptions
## Guides
- [Core Web Vitals guide](https://example.com/core-web-vitals-guide/): LCP, INP and CLS explained
- [WooCommerce vs Shopify](https://example.com/woocommerce-vs-shopify/): How to choose a store platform
## Optional
- [Blog](https://example.com/blog/): All articles
Keep descriptions factual. This file may be read by a machine and quoted; exaggerations are a liability.
How to add llms.txt to WordPress
Option 1: a static file
Upload llms.txt to the WordPress root (the folder with wp-config.php). Simple, but you must update it manually.
Option 2: generate it dynamically
Serve it through WordPress so it stays in sync with your content:
add_action( 'init', function () {
add_rewrite_rule( '^llms.txt$', 'index.php?mx_llms=1', 'top' );
} );
add_filter( 'query_vars', function ( $vars ) {
$vars[] = 'mx_llms';
return $vars;
} );
add_action( 'template_redirect', function () {
if ( ! get_query_var( 'mx_llms' ) ) {
return;
}
header( 'Content-Type: text/plain; charset=utf-8' );
echo '# ' . get_bloginfo( 'name' ) . "nn";
echo '> ' . get_bloginfo( 'description' ) . "nn";
echo "## Guidesnn";
foreach ( get_posts( array( 'numberposts' => 20 ) ) as $p ) {
printf( "- [%s](%s)n", get_the_title( $p ), get_permalink( $p ) );
}
exit;
} );
Flush permalinks after adding a rewrite rule (Settings → Permalinks → Save, or wp rewrite flush). Some SEO plugins now offer llms.txt generation too.
Option 3: on other platforms
On hosted platforms where you cannot place files at the root, check whether the platform or an app supports it, or use a redirect if allowed.
Writing a good llms.txt
- Lead with a precise summary: who, what, for whom. Reuse your entity description.
- Link to canonical URLs only.
- Prefer pages with substantial, unique content.
- Group logically: services, about, guides, policies.
- Keep it short; the point is curation.
- Update it when key pages change.
Caching and serving
- Serve as
text/plainortext/markdownwith UTF-8. - Make sure caching plugins do not serve an HTML error page at that URL.
- Check it is not blocked by robots.txt or a firewall rule for bots you want to allow.
How to check it works
curl -sI https://example.com/llms.txt
curl -s https://example.com/llms.txt | head -20
Expect a 200 status and your markdown content.
Common mistakes
- Treating llms.txt as a magic AI ranking switch and neglecting real content.
- Listing hundreds of links, which defeats the curation.
- Including claims that are not on the site.
- Letting it go stale after a redesign.
Who llms.txt is actually for
llms.txt is most useful when a person or tool asks an AI model to read your site on demand: a developer pasting your documentation into a coding assistant, an agent browsing your pricing page, or a tool that fetches a site to summarise it. In those cases a short, curated index helps the model find the right pages quickly instead of parsing menus, sliders and cookie banners.
It is much less clear that large search crawlers use the file. The llms.txt proposal is a community convention, not a standard adopted by Google. Google’s own guidance on AI features in Search says the same SEO fundamentals apply and no special file is required. So treat llms.txt as a cheap, low-risk addition, not a ranking tactic.
A decision checklist: should you add one?
- Yes, worth it: software products, documentation sites, agencies with detailed service and process pages, knowledge bases, and course sites where people often ask AI tools about you.
- Optional: local service businesses with a handful of pages. It takes twenty minutes, but expect little measurable effect.
- Fix other things first: sites with indexing problems, blocked resources or thin pages. A curated file pointing to weak pages does not help.
Example: llms.txt for a bilingual agency site
Sites in Egypt and the Gulf are often bilingual. One file can list both languages clearly, so a model does not mix them up. An illustrative structure:
# Example Studio
> Web design and SEO studio in Cairo serving clients in Egypt and the Gulf. English and Arabic.
## Services (English)
- [Website design](https://example.com/services/web-design/): process, deliverables, timelines
- [Technical SEO](https://example.com/services/seo/): audits and fixes
## الخدمات (العربية)
- [تصميم المواقع](https://example.com/ar/services/web-design/): المراحل والتسليمات
## Guides
- [Technical SEO checklist](https://example.com/technical-seo-audit-checklist/)
## Optional
- [Case studies](https://example.com/portfolio/)Notes on the example: the summary line states who you are, where you work and in which languages. Each link has a one-line description of what the page contains, not marketing copy. The “Optional” section signals pages a tool can skip if context is short, which matches the convention described on llmstxt.org.
llms.txt vs robots.txt vs sitemap.xml
| File | Purpose | Who reads it |
|---|---|---|
| robots.txt | Controls which paths crawlers may request | Search and AI crawlers that respect it |
| sitemap.xml | Lists URLs you want discovered and indexed | Search engines |
| llms.txt | Curated, human-readable map of your most useful pages | AI tools and agents that choose to fetch it |
The files do not overlap. llms.txt cannot grant or block access; if you want to control AI crawlers, that happens in robots.txt with the relevant user agents.
Maintenance routine
- Review the file whenever you add a service, change prices or retire a page.
- Check monthly that every linked URL returns 200 and is not redirected.
- Keep it short. If the file grows past a few dozen links, it is becoming a second sitemap.
- Check server logs for requests to
/llms.txt. That is the only honest signal of whether anything reads it.
FAQ
Will llms.txt get my site cited in ChatGPT or Google AI Overviews?
There is no public evidence that it does. Citations depend on being crawlable, indexed and genuinely useful. llms.txt can help tools that fetch it, and nothing more is promised.
Should I include every blog post?
No. Include the pages that best explain who you are and what you offer, plus a few cornerstone guides. Curating is the whole point.
Do I need llms-full.txt too?
Some sites publish a longer version with full page text. It suits documentation sites. For a business site, a well-written llms.txt is enough.
The fundamentals still decide visibility: crawlable pages, clear facts and helpful content. See how to rank in AI Overviews. Want llms.txt and entity markup set up for your site? Contact me.
Frequently asked questions
What is llms.txt?
llms.txt is a proposed convention: a markdown file at /llms.txt that summarises a website and links to its key pages, so large language models and AI tools can understand it quickly at inference time.
Is llms.txt an official standard?
No. It is a community proposal published in 2024. It is not part of an official standards body, and major AI search providers have not committed to using it as a ranking or citation signal.
What is the difference between llms.txt and robots.txt?
robots.txt tells crawlers what they may access. llms.txt does not control access; it offers a curated summary and reading list for AI tools.
What is llms-full.txt?
Some sites also publish llms-full.txt, a single file with the full text of key documentation, so tools can load everything at once. It is a common extension, not a requirement.
