5-step deployment guide
How to Create llms.txt
Create a curated llms.txt file by choosing important pages, writing clear summaries, validating the structure, and publishing it at /llms.txt.
LLMs.txt is an emerging proposed standard. Use this workflow as a practical guide, not as a ranking guarantee.
Workflow
Practical checks
Choose canonical pagesReview before publishing.
Write a short summaryReview before publishing.
Group URLs into sectionsReview before publishing.
- Move secondary links to Optional
- Upload to root path
- Verify public access
Implementation note
Keep the file curated.
Start with the pages that explain the product, service, documentation, policies, and examples. Exclude login pages, carts, search results, tag archives, duplicate pages, private URLs, and other low-value resources. Validate the file, then place it at your domain root as /llms.txt.
Common states to handle
Invalid or private URL
Block localhost, private networks, metadata endpoints, and non-http protocols.
Blocked by robots.txt
Respect robots rules and allow manual URL input when crawling is restricted.
Download success
Copy or download the file, then verify the public root path.