How to make your WordPress site readable by AI

Your website is written for people. Navigation menus, cookie banners, sidebars, related-post widgets, a footer with four columns of links. A human reader skips past all of it without noticing. An AI assistant fetching the same page has to work out which part of that markup is the article and which part is furniture, and it does not always guess correctly.

Making your site “readable by AI” means publishing a version of your content with the furniture removed. That is what llms.txt does, and on WordPress it takes about two minutes to set up. This post walks through the whole thing: what you are actually publishing, how to generate it, how to confirm it worked, and the handful of settings worth a second look.

If you have not met the format yet, start with what llms.txt is and who reads it. This post assumes you have decided to publish one.

What “AI-readable” means in concrete terms

It means two files at the root of your domain.

yoursite.com/llms.txt is an index. Your site name, a one-line description, then a structured list of your pages, posts, and products, each with a link and a short summary. An assistant that fetches this file gets an accurate map of your site in a single request.

yoursite.com/llms-full.txt is the same index with the full text of every page inlined underneath it, converted to Markdown. One fetch, complete content, no HTML to parse.

Both are plain Markdown. Here is roughly what the top of an llms.txt looks like:

# Bright Harbor Coffee

> Small-batch roasting in Portland since 2011

## Pages
- [About](https://example.com/about): Who we are and how we source our beans
- [Wholesale](https://example.com/wholesale): Pricing and terms for cafes and offices

## Posts
- [How we choose a farm](https://example.com/blog/choosing-a-farm): Our sourcing criteria, explained

Nothing exotic. That readability is the entire point.

Step 1: install the plugin

Search for Project LLMs.txt under Plugins → Add New, install, and activate. It is free on WordPress.org.

Two requirements worth confirming before you start. You need WordPress 6.4 or higher and PHP 8.1 or higher, which covers most reasonably current hosting. If you want physical files written to your site root, PHP also needs write permission to that directory. Most hosts allow this. If yours does not, there is a setting for that case and we will come back to it.

After activation, a Project LLMs.txt item appears in your admin sidebar.

Step 2: tell AI how to introduce your site

Open the plugin. On first run the Generate tab shows a short setup card rather than the usual dashboard, and step one on that card is the only part that needs your judgment.

Site name and tagline are pre-filled from your WordPress settings. They become the heading and the summary line at the top of your llms.txt. If your tagline is still “Just another WordPress site,” fix it here, because that sentence is the first thing an AI assistant reads about you.

Extended site description is an optional paragraph underneath. This is your one chance to state plainly what your site is, who it serves, and what makes it different. Two or three sentences is plenty. There is a live preview beside the fields showing the top of your llms.txt updating as you type.

Step two is a scan of your site, listing each content type it found with an item count. Read it. If something you expected is not on the list, that is the moment to notice. Posts and pages are included by default, and if you run WooCommerce or have custom post types holding real content, you can add them under Settings → Content Types before generating.

Step three is a preview of the files you are about to get. Nothing to do there.

Step 3: click Generate Now

Click Generate Now at the bottom of the card.

The plugin walks your published content, reads each item, converts the HTML to Markdown, and assembles the two files. On a small site this finishes in seconds. On a large catalog it takes a little longer and shows a progress bar.

What is happening under the hood is the interesting part. For each page, the plugin reads your published content directly, including block output, shortcodes, and page-builder markup. For content your theme renders outside the editor, it can scrape the rendered page instead and use a Readability-style algorithm to find the main content zone, the same general approach browsers use for reader mode. Then it converts what it found to Markdown and drops it into the template for that content type.

You do not have to configure any of that. The default extraction mode is set to auto and handles both cases.

Step 4: verify the files are live

Open yoursite.com/llms.txt in a new tab. You should see your site name, your description, and your content listed out. Do the same for yoursite.com/llms-full.txt.

The plugin header gets a View llms.txt button once a file exists, and the Generate tab shows a file panel with each file’s size and when it was last built.

Read the output rather than glancing at it. This is the version of your site an AI assistant will work from, so it is worth checking that your important pages are present, the descriptions read sensibly, and nothing obviously internal has leaked in.

One thing that surprises people: if you open the file directly in a browser and see garbled accented characters, the file is almost certainly fine. Some web servers serve .txt files without declaring a character set. The file is UTF-8, and AI tools read it correctly. If it bothers you, ask your host to add AddDefaultCharset UTF-8, or switch the plugin to Live Endpoint mode, which always declares the charset.

Step 5: the settings actually worth revisiting

Once you have a file you are happy with, four settings deserve a look.

File mode. Physical files are written to your web root and served directly by your web server, which is the fastest option and the default. Live Endpoint mode serves the same content dynamically through WordPress rewrite rules instead. Use it if your root directory is not writable, or if you are on a host with a read-only filesystem.

Exclusions. Some pages have no business in a machine-readable summary of your site. Thank-you pages, internal landing pages, anything gated. The exclusion picker in Settings lets you remove them. If WooCommerce is active, the plugin already excludes Cart, Checkout, and My Account automatically, since those are checkout mechanics rather than content.

Your SEO plugin’s noindex settings are honored. If a page is marked noindex or nofollow in Yoast, Rank Math, SEOPress, AIOSEO, or Slim SEO, it is left out of the AI files too. Your existing exclusions carry over instead of quietly leaking through a new surface.

The robots.txt pointer. On by default, this appends a short block to your robots.txt advertising the two files:

# AI Assistants
User-agent: *
Allow: /llms.txt
Allow: /llms-full.txt

It is a signpost, not a requirement. Nothing breaks without it. See llms.txt vs robots.txt vs sitemap.xml for how the three files relate.

What to do next

If you run a store, the WooCommerce output is where this gets substantially more useful. Products are exported with structured price, SKU, stock status, variation pricing, reviews, and related products rather than as a wall of theme markup. That is worth its own walkthrough: llms.txt for WooCommerce.

If you want to see one page’s Markdown, the Generate tab lists every item with a preview, so you can check exactly what a given page turned into before you worry about the whole file. Editing that output by hand, or changing the templates the plugin renders from, is available in Pro.

If you manage more than one site, there is a WP-CLI command set. wp llmstxt generate and wp llmstxt status cover most of what a deploy script or a cron job needs.

And whatever you run, plan for the file going stale. This is the part people miss. The moment you publish a post, change a price, or update a policy page, your llms.txt describes a site that no longer exists. A confidently wrong machine-readable file is worse than none at all. You can regenerate manually whenever you make changes, which is genuinely fine for a site that rarely moves. For sites that change weekly or daily, automatic regeneration on publish is the Pro feature that exists for exactly this problem.

That is the whole setup. Install, confirm your site info, click Generate, read the output. Everything after that is refinement.

Project LLMs.txt generates llms.txt and llms-full.txt from the content you have already published, with first-class WooCommerce support and WP-CLI commands. The free version is on WordPress.org.