llms.txt
Definition
llms.txt is a plain text file at the root of a website that lists the pages worth reading and links to clean markdown versions of them, so an AI crawler can find the readable content quickly.
The file follows the pattern set by robots.txt and sitemap.xml: a predictable location holding a short machine readable summary. Where robots.txt says what may be fetched, llms.txt says what is worth fetching and where the plain version lives.
It is a convention rather than a standard, and no crawler is obliged to read it. It costs almost nothing to publish when the content is already structured.