llms.txt
A text file placed at a site’s root, designed to summarize its content in plain language for language models.
Definition
llms.txt is a Markdown-format file, placed at a domain root (example.com/llms.txt), that presents in structured natural language what a site is, what it offers, and points to its most important pages or resources. It was proposed in late 2024 by Jeremy Howard (answer.ai) as a community convention, without being an official standard enforced by AI engines.
Key takeaways
- It follows a simple structure: an H1 title, a blockquote summary, context paragraphs, then H2 sections listing annotated Markdown links.
- It replaces neither robots.txt (crawl permissions) nor sitemap.xml (exhaustive URL listing): its role is to point a model to what matters, not to manage access or exhaustiveness.
- No major AI engine (ChatGPT, Gemini, Perplexity, Claude) currently confirms using it systematically for crawling or grounding.
- Google checks for its presence in its experimental Lighthouse category dedicated to agentic browsing, alongside audits on WebMCP and machine accessibility.
Explanation
The value of llms.txt is mostly indirect: cheap to create, it forces a site to clearly state what it offers and prioritize its key pages, an exercise that benefits the general content structuring useful for grounding.
Its status remains that of an emerging convention rather than a guaranteed ranking lever. Its presence nonetheless becomes a technical readiness signal that Google watches as part of agentic browsing.
Go further
llms.txt: how to build it and its impact on AI visibility
The full article with the detailed structure and an honest state of play of its impact.
Agentic browsing
Google’s Lighthouse category that checks, among other things, for the presence of an llms.txt.
Grounding
The complementary mechanism: how an AI engine picks and cites its sources.
AIglebotBetaAIglebot helps companies track their visibility, reputation and recommendation in ChatGPT, Gemini, Perplexity and Claude.