llms.txt generator

The idea was put forward in 2024 as a way to help language models use a website at answer time. Web pages are full of menus, scripts and layout. A clean Markdown file skips that and says what the site is and where the useful pages live.

The proposed format is simple. It starts with a title, then a short summary in a quote block, then optional notes, then sections of links with a line describing each. An optional section can list pages that matter less.

It is not the same as robots.txt. Robots.txt tells crawlers what they may fetch. llms.txt only describes. It does not grant or deny access, and it does not replace a sitemap.

The page above is honest about status, so this section is practical. Use the generator when you have a reason that does not depend on a ranking benefit. Good reasons are short. You publish documentation and want coding assistants to find the right pages. You want one approved description of your business that your team, your profiles and your site can all be checked against. You are running a measured test and want to record the date you published. Or you simply want a tidy, readable index of your best pages and the file costs you twenty minutes.

Skip it, for now, if the real work is elsewhere. If your robots.txt blocks answer crawlers, if your key pages render only with JavaScript, or if your service pages do not answer buyer questions, fix those first. Those problems cost you visibility in ways an overview file cannot offset. A file that points to blocked or thin pages helps nobody.

The generator builds the file entirely in your browser. You type a name, a summary, optional notes and sections of links. It writes the Markdown as you type. Nothing is uploaded, and there is no account.

The proposal comes from Jeremy Howard and was dated 3 September 2024. The version published at llmstxt.org describes a Markdown file with a fixed order. As of October 2026, the proposal states that only the first element is required.

First, an H1 heading with the name of the project or site. This is the only required element. Second, a blockquote with a short summary that holds the key information needed to understand the rest. Third, zero or more detail sections of any Markdown type except headings, such as paragraphs or lists, that explain the site and how to read the linked files. Fourth, zero or more H2 sections that hold file lists. Each list item is a link in the form name and URL, optionally followed by a colon and a note. Finally, an H2 section called Optional is, by convention, where secondary links go, so a tool with a small context window can skip them.

The generator follows this order. It starts with your name as the H1, adds your summary as a quote line, adds any notes as plain text, then writes one H2 per section you create, with one list item per link. It writes each item as a Markdown link, followed by a colon and your note when you add one.

Imagine an IT services company in Dubai that sells annual maintenance contracts. Its summary line says it provides managed IT support and annual maintenance for small and mid sized offices in Dubai, with response times stated in the contract. Under a section named Services, it lists three pages: the AMC overview, the managed support page and the network monitoring page, each with a note that says who the service is for and what is included.

Under Pricing and process, it lists the page that explains how a quote is built and the page that explains onboarding steps. Under Guides, it lists two explainers on what an AMC covers and how to compare contracts. Under Optional, it puts the news archive and the privacy policy.

The result is about twelve lines. It holds one approved description, a clear route to the pages that answer buyer questions, and nothing the company would not want read aloud. It does not promise that any engine will use it. The firm also checks that each listed page is indexed and loads its text without scripts, since a guide that points to a hollow page is worse than no guide.

As of October 2026, the proposal at llmstxt.org is a community proposal, not a formal standard. Google's documentation on AI features in Search says that you do not need to create new machine readable files, AI text files or markup to appear in AI Overviews or AI Mode, and that pages need to be indexed and eligible to show a snippet. The page does not name llms.txt, so we read that as a general statement and not a specific ruling.

We have not found OpenAI, Anthropic or Perplexity documentation that says their crawlers use llms.txt to rank or choose sources. Absence from documentation is not proof of non use. It does mean you should not build a plan on it. Some developer tools and coding assistants can read a file you point them to, which is a real but narrow use.

If you want to test for yourself, do it properly. Record your answers on a fixed set of buyer prompts before you publish. Publish. Change nothing else for four to six weeks. Test again. If anything moves, you still cannot be sure the file caused it, because engines vary from run to run. Our guide on measuring GEO performance explains how to treat such tests honestly.

It cannot check that your links work. It validates only that the address begins with https:// and that the title and URL are present. It does not fetch the pages, so a mistyped path will pass. Test each link by hand.

It cannot judge your wording. It will write whatever you type. Two weak sentences become a weak file.

It does not create a full Markdown version of each page. The proposal also suggests offering clean Markdown copies of pages at a matching address with an md ending. That is a separate piece of work and often needs development help, so it is outside this tool.

And it cannot make any engine read the file. Publishing is a request to be understood, not a command.

For the longer discussion, read llms.txt: do you need one. For the file that actually controls access, use the AI crawler robots.txt generator. For structured data that supports the same pages, see schema markup for AI search.

In the GEO Pilot, llms.txt is on the list of Week 2 fixes only if it is useful for your site, and we say so plainly when it is not. The work that matters more is access, structured data and answer ready pages. The Pilot is $999 fixed, one time, delivered inside 30 days, and we make no promise of rankings, citations or traffic. See the GEO Pilot page.