llms.txt checker
AI assistants and agents increasingly read sites the way crawlers read robots.txt — and /llms.txt is the file that maps them straight to your key pages instead of making them guess. Enter your domain and we fetch it and grade it: no file at all, an HTML soft-404 answering for it, an empty file, or a file with no links to follow.
The file, the problems, the exact lines
Six findings, the same ones the £7.99 audit flags
This checker runs the llms.txt portion of our real audit engine — one of the agent-experience checks behind the AI-Readiness category in every report we sell.
1 · No llms.txt at all
Nothing answers at /llms.txt. An agent that probes for it finds nothing to read and falls back to crawling and guessing what matters. Flagged as an error — one markdown file at your domain root fixes it.
2 · The URL serves HTML, not text
The classic SPA trap: the server answers every path with the app shell, so /llms.txt returns a web page. Agents that probe it get markup instead of a content map — the same practical effect as no file. Flagged as an error.
3 · The file exists but is empty
An empty llms.txt signals intent that was never finished, and gives an agent nothing. Flagged as a warning.
4 · No markdown links to your content
The whole point of the file is a curated map: [page title](https://full-url) lines, grouped under ## sections. Without links there is nothing to follow. Flagged as a warning.
5 · No H1 or ## structure
The convention starts with one '# Site name' line and groups links under ## headings — that structure is what makes the file skimmable by a machine. Missing structure is flagged as an informational note.
6 · Wrong content-type
The file should be served as text/markdown. Some clients sniff the body anyway; setting the type removes the ambiguity. Flagged as an informational note.
What a good llms.txt looks like
Small, structured, honest. An H1 naming the site, a one-line summary, and sections of markdown links to the pages that matter.
- Line 1 is the identity line. One H1, the site's name — the first thing an agent reads.
- Line 2 is the pitch. One blockquote sentence: what this site is, for whom.
- Lines 4–10 are the curated map. ## sections, absolute-URL markdown links, the pages you would want quoted. Not the sitemap — the shortlist.
- Keep it maintained. A stale llms.txt is worse than none: an agent repeats your outdated pitch with confidence.
llms.txt is 1 of 260+ checks
The checker above is free and instant. The full £7.99 audit runs all of it: 260+ checks across 21 categories on up to 25 pages, an AI-written plain-English report, delivered by email in ~30 minutes. What it costs · read a real sample report.
The full audit
Everything on this page plus the other 250+ checks, in one ordered fix-list.
- llms.txt presence, structure and content quality
- AI-crawler access: GPTBot, ClaudeBot, PerplexityBot & friends in robots.txt
- Structured data (JSON-LD) presence and validity
- Agent-discovery files: llms-full.txt where present, API catalogs
- Agent-facing endpoints: MCP/A2A reachability where offered
- Everything else: SEO, crawlability, security, accessibility, performance
— Not ours to claim
What no external tool can do:
- See inside Search Console — crawl stats and indexing requests live behind your Google login; the report names the exact screen to open
- Edit your site for you — the fix is yours to apply (or ours: we do fixes too); the report gives the exact lines
- Promise rankings or AI citations — nobody honestly can; what the report guarantees is that nothing on your site mechanically blocks the machines that read it
Also honest: this checker looks at one slice. A passing slice with failing pages elsewhere is exactly what the full crawl is for.
Fair questions, straight answers
What is llms.txt?
A proposal (from the AI tooling community, 2024) for a markdown file at your domain root that gives AI assistants and agents a curated map of your site: the site name, a one-line summary, and markdown links to your key pages grouped under headings. The idea is that an agent reading one small file gets your real pitch and your real pages, instead of crawling thousands of URLs and guessing.
Does Google or ChatGPT actually use llms.txt?
Adoption is mixed and no major engine has committed to it as a ranking input — treat any tool that promises AI citations because of llms.txt with suspicion. What is true: AI agents increasingly probe well-known paths, the file costs an hour to write and keep current, and it makes your agent-readiness deliberate instead of accidental. That is exactly how the audit scores it: a readiness signal, not a ranking lever.
What should go in my llms.txt?
An H1 with the site name, a blockquote one-liner saying what the site is and for whom, then ## sections with absolute-URL markdown links to the pages you would want an AI to read and quote: about, pricing, key products or services, documentation, contact. It is a shortlist, not the sitemap — 10 to 30 links beats 5,000.
Is llms.txt the same as robots.txt for AI?
They are opposites that work together. robots.txt tells crawlers what they may NOT fetch; llms.txt tells agents what they SHOULD read. A site can block training crawlers in robots.txt and still publish llms.txt for assistant-style agents it wants to answer from its pages — and both decisions are visible in the full audit so they stay deliberate.
One file for the machines. 259 more checks for everything else.
Enter your URL and pay securely — your report PDF lands in your inbox in ~30 minutes,
with llms.txt, AI-crawler access, structured data and 250+ other checks run and explained.