LLMs.txt

Overview

LLMs.txt publishes a machine-readable guide for AI systems at llms.txt and can also generate extension files for more specialized output.

Admin path: Airygen SEO -> Settings -> LLMs.txt

This module is useful when you want more control over how AI systems discover and prioritize your content.

Settings

The page is divided into Settings, Extensions, and Preview.

Settings tab

Use this tab for the main llms.txt file.

• Enable llms.txt output publishes the base file.
• Index strategy offers Selections only, Selections + auto, or Auto only.
• Use markdown post links switches post links to markdown-style links when Markdown for Agents supports that post type.
• Add to sitemap includes the base file and enabled extensions in the sitemap index.
• Auto-add Topic Cluster groups can add Topic Cluster content automatically when that module is enabled.
• Auto-loaded section title names the automatic content section.
• Custom declaration adds optional site-level instructions near the top of the file.
• Clear cache forces the module to rebuild output after major content changes.

The tab also includes scope and filtering controls.

• Post types decides which post types are eligible for auto-selection.
• Exclude noindex removes noindex content from automatic inclusion.
• Exclude password-protected removes password-protected content.
• Minimum word count helps exclude very short content from automatic selection.

Curated content is managed through Selections.

• Add selection creates a curated section.
• Each selection can include a Title, Description, and chosen posts.
• The selection modal includes Settings, Selected Posts, and Search posts.
• Hidden selections staging area stores selections that should not appear in output right now.

Extensions tab

Use this tab when you want more than one AI-facing file.

• Add extension creates a separate file configuration.
• Each extension has its own Title, Description, Path, Filename, enable toggle, and Custom declaration.
• Each extension has its own curated selection manager.
• Hidden selections can also be staged inside each extension without being published.

Preview tab

Use this tab to inspect the generated output before testing it live.

• You can choose the base file or a specific extension as the preview target.
• Preview llms.txt generates the current output using your latest settings.
• The Output area shows the generated text for review.

How to Use

1. Open Airygen SEO -> Settings -> LLMs.txt.
2. Turn on Enable llms.txt output.
3. Choose an Index strategy that matches how curated you want the file to be.
4. Decide whether Use markdown post links, Add to sitemap, and Auto-add Topic Cluster groups should be enabled for your workflow.
5. Fill in Auto-loaded section title and Custom declaration if you want clearer guidance for AI systems.
6. Choose the post types that are eligible for auto-selection and set the exclusions you want.
7. Build curated Selections for the pages and posts you want to highlight intentionally.
8. Move unused selections into the Hidden selections staging area instead of deleting them if you may reuse them later.
9. If you need additional AI-facing files, open Extensions, add an extension, and curate its own selections.
10. Open Preview, choose the base file or an extension, and review the generated output carefully.
11. If the preview still reflects old content after major changes, use Clear cache and preview again.
12. Test the live file paths after saving, including /llms.txt and any enabled extension paths.

Check these items after saving:

• The base file contains the sections you intended to publish.
• Hidden selections are not appearing in the output.
• Exclusions such as noindex and password-protected content are working.
• Extension paths and filenames open successfully.
• Preview output matches the live files you expect to expose.

SEO Benefits

LLMs.txt does not replace standard SEO files, but it gives AI systems a cleaner and more intentional map of your content. That can support better AI discovery and reduce the chance that low-value pages become the main machine-readable entry points for your site.

User Cases

Documentation and knowledge sites

Before: A site has many pages, but only a smaller set should be highlighted for AI systems first.

After: LLMs.txt lets the team curate high-value pages into selections instead of exposing everything equally.

Teams that want multiple AI-facing files

Before: One generic llms.txt file is not enough because the site needs different AI-facing entry points for different content groups.

After: Extensions lets the team publish additional curated files with their own paths and selections.

FAQ

Does LLMs.txt replace the XML sitemap?

No. It is a curated instruction file for AI systems, not a sitemap replacement.

Can I publish more than one AI-facing file?

Yes. Use Extensions to create additional files with their own paths, filenames, and selections.

Why is Use markdown post links unavailable on some sites?

That option depends on Markdown for Agents. If that module is not enabled for the relevant content, markdown post links cannot be used there.