# How to make your website usable by AI agents A free skill for Claude Code or any agent: it makes your site readable and usable by AI agents, finds the questions your site leaves unanswered, and tests it all with a real agent. By Tyler Malone · Published Oct 10, 2026 · Updated Oct 10, 2026 ## Why I bothered Some of the people looking for a business like mine never see my website. They ask ChatGPT or Claude, and an agent reads the site for them. So I set out to make contraptionsoft.com easy for an agent to read and use, and then I did the part most guides skip: I had an agent actually use it, the same way a customer's agent would. It worked, mostly. The agent started from one file, figured out what we do, found a past job that matched its task almost exactly, and sent a real inquiry through our contact form that landed in our CRM. It also found a bug I'd shipped that broke the site for exactly the agents I was trying to help, and every normal test I ran had missed it. I packaged everything I learned as a skill. Give it to your AI agent and it does the same thing for your site, including the testing. Not into terminals? [We'll do it for you](/contact). Just mention the agent-ready skill. ## What I changed None of it is new tech. It's four small things that tell an agent where to look and how to read what it finds. - **An llms.txt file.** It's a plain-text page at `/llms.txt` that summarizes the business and links to every page, written for a model instead of a person.[1] Ours is generated from the same lists the site uses, so a new post shows up without anyone editing it. - **Markdown versions of every page.** An agent can add `.md` to any URL, or ask for Markdown in its request, and get clean text without the layout and styling mixed in. - **Instructions for the contact form.** The llms.txt file explains exactly how to send us an inquiry, with the fields it needs, and tells the agent to do it only when its person asked and with that person's real name and email. - **A Content-Signal line in robots.txt.** It's a short line that says what our content may be used for: search, answering questions, and training.[2] More on training below. ## The agent found a bug I couldn't I tested all of it the normal way, with a browser and with curl, and everything passed. Then I had Claude use the live site through its own web fetch tool, and the first file it asked for, llms.txt, came back as a 404. The reason is a little nerdy, but it's the most useful thing in this post. Agents that prefer Markdown say so in every request, and I'd set the site up to send any request asking for Markdown to the Markdown version of the page. "Any request" turned out to include llms.txt itself, the sitemap, images, the files you can download, and the contact form. So for agents that prefer Markdown, which includes Claude's fetch tool, llms.txt didn't exist and the contact form returned an error. Browsers never ask for Markdown, so every test I ran in a browser looked perfect. The fix took a few lines: only actual pages get the Markdown treatment, and everything else is left alone. The lesson is the part that matters. **You can't test an agent-ready site with a browser.** You have to test it with an agent, using the same tools a real one uses. The skill does that before it calls anything done. After the fix, I ran the whole thing again. The agent read llms.txt, followed it to the right pages, answered a made-up customer's questions about automating purchase orders in Zoho, and sent one inquiry clearly labeled as a test. It showed up in our CRM. ## The second site needed something different Before posting this, I ran the skill on a site we built for a client, [Malone Excavation](https://malonedigs.com). It's a single page with a phone number and no contact form, which is how a lot of small business sites look. Agents could already read it fine. The content is in the page, and nothing was blocking them. When I asked an agent to help a homeowner in Bryant get a French drain installed, it found that Malone does French drains, that you deal directly with the owner, and that estimates are free. What it couldn't find out was whether Malone works in Bryant (the site says Saline County, but it never names the towns), what a French drain might cost, or how to reach them other than by calling. None of that is a technical problem. Markdown files and llms.txt can't fix a question the site never answers. For most small business sites, the biggest improvement isn't anything technical. It's writing down the answers people ask on the phone every day: the towns you serve, a rough price range, your hours, and how to reach you. The skill now looks for those gaps first, gives them to you as a list, and skips the heavier setup when a site doesn't need it. ## Check Cloudflare, or whatever sits in front of your site If your site goes through Cloudflare, like mine does, the settings there can override everything above. Cloudflare can block AI traffic before it ever reaches your site, and it can rewrite your robots.txt. It now splits AI traffic into three kinds you can allow or block separately: search crawlers, agents fetching a page live for a person, and training crawlers.[3] Since September 15, 2026, new sites that show ads default to blocking agents on pages with ads and disallowing training.[4] If your goal is for agents to read your site and send you customers, the agent setting is the one you can't get wrong. Your code can be perfect and an agent will still get a block page. You can't fix these from your site's code, and an agent testing from its own machine can't see them, so the skill gives you a short checklist for your dashboard. Other hosts and CDNs have similar settings under different names. ## Should you let AI train on your site? This is a real choice, so here's what it means. When an agent reads your site to answer someone's question, it reads the page live, uses it, and moves on. Training is different. Training means your content goes into the data a company uses to build its next model, so the model learns about your business and remembers it without needing to look anything up. The Content Signals standard keeps the two separate: `ai-input` covers the live reading, and `ai-train` covers training.[2] Some reasons to say no: your content is your product (a publisher, a course, original research), you don't want it used in ways you can't control, or you'd rather be paid for it. Those are fair reasons, and it's your content. I say yes. When someone asks a model who builds custom software in central Arkansas, I want the model to already know about ContraptionSoft, without having to search for it. The only way to get into the model's memory is to be in what it trained on. For a small local business that wants to be found, giving that content away is a trade I'm happy to make. Either way, I'd allow the live reading. Blocking it means an agent helping a customer can't read your site at all, and it'll send them somewhere else. ## What this won't do I'll be straight about this: nobody knows yet how much of this pays off. Google says you don't need special files or markup to show up in its AI answers,[5] and plenty of agents never look for llms.txt at all. What's definitely true is that agents read websites today, some prefer Markdown, and a site that answers their questions and lets them reach you gets the lead while a site that doesn't gets skipped. The setup is a small one-time job, so I'd rather be early. If you sell products online, agents buying things is a separate and much bigger project. I wrote about [who should build agent checkout and who should wait](/notes/agentic-commerce-checkout). If you sell a service, this note covers the part that applies to you right now. ## What your agent will do 1. **Learn how your site is built** and what a visitor can do on it. It also asks whether to allow training and which actions an agent may take for someone. 2. **Use your site the way an agent does**, then show you what works, what doesn't, and which questions your site can't answer. You decide what to fix. 3. **Update robots.txt** and give you a checklist for Cloudflare or your host. 4. **Add an llms.txt file**, generated from your site so it doesn't go out of date. 5. **Add Markdown versions of your pages**, but only if your site is big enough to need them. 6. **Document what an agent can do** on your site, like sending an inquiry. 7. **Check the structured data** that tells search engines and agents what your business is. 8. **Test everything with real agent requests** before it ships, then again on the live site. It asks before sending any real form, and it labels that request as a test. It stops for your OK before changing anything that matters, and it never makes up prices, service areas, or other facts to fill a gap. ## What you'll need - An AI agent that can edit code and run commands, like Claude Code - Your website's code, or access to wherever it's built - Access to your Cloudflare or hosting dashboard, if you use one ## Get the skill For Claude Code on Mac or Linux: ```bash mkdir -p ~/.claude/skills/agent-ready-site && \ curl -fsSL https://contraptionsoft.com/skills/agent-ready-site/SKILL.md -o ~/.claude/skills/agent-ready-site/SKILL.md ``` Then open your website's code and say: **"make my site agent-ready."** On Windows, or with any other agent, point it at `https://contraptionsoft.com/skills/agent-ready-site/SKILL.md` and ask it to install the skill and follow the steps. Here's exactly what you're installing: - [SKILL.md](https://contraptionsoft.com/skills/agent-ready-site/SKILL.md) --- *Want us to make your site agent-ready, or build a site that answers what your customers actually ask? That's what we do. See [web design](/web-design-benton-ar) or [get in touch](/contact).* ## Sources 1. [llms-txt, The /llms.txt file](https://llmstxt.org) 2. [Cloudflare, Giving users choice with Cloudflare's new Content Signals Policy](https://blog.cloudflare.com/content-signals-policy/) 3. [Cloudflare, New options to manage AI traffic](https://developers.cloudflare.com/changelog/post/2026-07-01-ai-traffic-options/) 4. [Cloudflare, Have it both ways: stay discoverable in search while disallowing AI training](https://blog.cloudflare.com/accountable-mixed-use-ai-crawlers/) 5. [Google Search Central, AI Features and Your Website](https://developers.google.com/search/docs/appearance/ai-features)