Some red marks are must-fixes. Some are theatre. Here is how to tell the difference before you ship anything.

An agent-readiness scanner will hand you a list of things your website is missing. Most of that list is real. Some of it is a trap, and the skill everyone needs now is telling the two apart. We ran one on a website we manage, shipped the parts that mattered, and deliberately skipped the rest. Here is how we decided which was which.

Key Takeaways

What Does an Agent-Readiness Scanner Actually Check?

It checks whether an AI agent can find your site, read it, and operate it. Tools like Cloudflare’s IsItAgentReady.com run a set of checks across several categories and return a scored report with specific fixes.

For practical decision-making, we grouped the scanner’s checks into two broader layers. The first is discovery and content: can an agent find your pages, read them cleanly, and understand what you offer. The second is capability and access: can an agent call your tools, log in, and pay.

Those two groups are not equally important for every site, and that is the whole point. A general scan can test a coaching site and a commerce platform against many of the same capabilities, even though they do not need the same things. Cloudflare now lets you narrow the scan by site type or choose individual checks, but the default report can still surface standards that are irrelevant to your business.

What Did We Decide Was Worth Doing?

The machine-discovery layer, because that is what helps an agent find and read the site regardless of what the site sells. This is the part almost every business benefits from, and it is where we spent the effort.

The foundation came first, and it is the part that helps nearly everyone: a clean robots.txt, a current sitemap, crawlable content, and clear link metadata so an agent can find its way around. On top of that we added two emerging content pieces. Markdown content negotiation, so an agent that asks for a clean text version gets one. And Content Signals in robots.txt, so we declare our preferences for how participating AI crawlers may use the content for search, AI input, and training.

We also wired in the callable-capability standards, but only because the site genuinely exposes callable capabilities. The site has public, machine-callable endpoints, none of which require authentication, so the API catalog documents APIs that genuinely exist. We also published an MCP Server Card so compatible clients can discover the site’s MCP server and the capabilities it exposes. We added WebMCP, an emerging browser API that lets supported agents invoke tools a page exposes, and an Agent Skills index, so compatible agents can discover reusable instructions and workflows connected to the site.

The honest test we applied to every one of these was simple: does it describe something the site actually does? Each of them did. None required faking anything, and that is the only reason we published them.

What Did We Deliberately Skip, and Why?

On the same site, we deliberately left the entire authentication and commerce layer alone, because that site does not have the things those standards exist to protect. This is where most scanner results go to die, and skipping them was the right call, not a shortcut.

The scanner flagged OAuth discovery metadata and OAuth Protected Resource metadata as missing, and it flagged the site’s existing auth.md because it does not describe an agent-registration flow. The OAuth metadata helps agents discover how to obtain authorization for a protected API, MCP server, account, or other private resource. The auth.md proposal describes how an agent can register for a service. Because this site exposes nothing private and has no agent-registration flow, adding that information would describe doors that do not exist.

Here is the part worth internalising, because it is the whole lesson. Whether you skip these is not a fixed rule. It depends entirely on what the site actually does. A membership platform, a SaaS app, or anything with a login and a protected API genuinely needs OAuth discovery, and leaving it out would be a real gap. A brochure site, a blog, or a service business with no login does not, and adding it would be theatre. The same red mark is a must-fix on one site and safe-to-ignore on the next. The scanner cannot tell which you are. You can.

The commerce layer works the same way. We left the agentic commerce standards alone because that site is not a storefront, so a commerce protocol would be scaffolding around an empty room. A real online shop that wants agents to buy on a customer’s behalf would treat those exact standards as essential. Same checks, opposite verdicts, decided entirely by what the site actually does.

What Is the One Rule That Matters Most?

Never fake a standard to make a scanner happy. A green check that describes something your site cannot actually do is not a win, it is a lie that a real agent will eventually call.

Picture publishing OAuth metadata on a site with no protected resource. An agent reads it, believes there is something it can obtain authorization to access, tries, and fails because the promised resource or authorization flow does not actually exist. You have not improved your agent-readiness. You have created a trap that wastes an agent’s time and makes your site look broken to the exact audience you were trying to impress.

The honest version is simpler and stronger. Publish the standards that describe what your site genuinely does, and leave out the ones that would describe things it does not. An empty marker is worse than an absent one, every time.

How Should Most Businesses Read Their Own Scan?

As a menu, not a report card. The raw score has limited meaning by itself, because it measures your site against a wide range of capabilities a website could theoretically expose, most of which you will never need.

Read it in layers. Robots.txt, sitemaps, crawlable HTML, and clear site structure are foundational, and worth fixing if you fail them. Markdown negotiation, Content Signals, Agent Skills, Server Cards, and WebMCP are emerging enhancements whose adoption and client support remain limited. They can be useful early bets when the underlying capability is real, but they are safe to skip when it is not. The auth and commerce checks come last, and you can ignore them entirely unless your site has logins, private data, or a checkout.

If you are a coach, a consultant, or a small business with a content site, the foundational discovery layer may be most of your job, and a scanner that dings you for missing commerce protocols is testing you for a business you are not running. Fix what applies. Skip what does not. Fake nothing.

What Did This Look Like on a Site We Work On?

We put this framework to the test on that same website. After the discovery, content, and callable-capability work went in, the site scored 71 and reached Level 5, which the scanner labels Agent-Native. The number was encouraging, but it mattered far less than the pattern underneath it.

Content passed completely. Bot access control, including AI bot rules and Content Signals in robots.txt, also passed completely. Discoverability was almost clean, with robots.txt, the sitemap, and link headers all passing.

Four of the seven API, authentication, MCP, and skills checks passed too: the API Catalog, MCP Server Card, Agent Skills index, and WebMCP. Every green result described something the site genuinely exposes.

The remaining authentication checks stayed red for expected reasons. OAuth discovery and OAuth Protected Resource metadata were absent because the site has no protected resources for them to describe. The scanner also flagged the site’s existing auth.md because it does not include agent-registration instructions. We deliberately chose not to add metadata just to turn those results green, because the site has no agent-registration flow to document.

One discovery check deserves a different response. DNS-based AI discovery, known as DNS-AID, came back missing. Unlike the authentication checks, this one is worth evaluating for this site, because the site does expose genuine agent-facing capabilities and DNS-AID sits in the discovery layer that helps agents find them.

So we are implementing the DNS-AID discovery records as part of our ongoing work, and we will publish a follow-up showing the updated results. That gives us a chance to measure something more useful than a higher score: whether the change makes any real difference to how agents discover the site.

That is the whole point in miniature. Some red marks represent capabilities a site does not need. Others reveal work that is genuinely worth doing. The skill is not turning every result green. It is knowing which recommendations reflect your reality and which do not.

Frequently Asked Questions

Q: My agent-readiness score is low. Is that bad?
A: Not necessarily. The raw score has limited meaning by itself, since general scanners test against a wide range of agent capabilities, including commerce and authentication standards most sites do not need. Read the individual checks rather than the number, and scope the scan to your site type where the tool allows it.

Q: Do I need OAuth on my website for AI agents?
A: Only if your site exposes a protected resource that an agent would need authorization to reach, such as a private API, a protected MCP server, or an account. OAuth discovery and protected resource metadata help an agent find out how to get that authorization. If there is nothing private to reach, they do not apply, and adding them would describe access that does not exist.

Q: Should I add every standard a scanner recommends?
A: No. Add the standards that accurately describe what your site genuinely does. Skip the ones built for capabilities you do not have, like commerce protocols on a site with no checkout.

Q: What happens if I publish a standard my site cannot actually support?
A: You mislead the agents that read it. Empty or fake metadata tells an agent your site can do something it cannot, which leads to failed interactions and makes your site look unreliable to the systems you wanted to reach. An absent standard is safer than a false one.

Q: Which agent standards should a normal business site prioritise?
A: Start with the foundations: a clean robots.txt, a current sitemap, crawlable content that agents can reliably retrieve and read, and clear structure. Those help almost every site. Emerging additions like Markdown negotiation, Content Signals, MCP, WebMCP, and Agent Skills are worth it only when the underlying capability is real, and auth standards only when your site has protected access to expose.

Agent-readiness scanners are useful the way a full-body checklist is useful, as long as you remember that not every item is meant for you. The sites that get this right will not be the ones chasing a perfect score. They will be the ones that shipped what was true and had the discipline to leave the rest blank.AI Visibility Studio helps websites structure content so AI systems can find it, understand it, cite it, and actually use it when generating answers. aivisibilitystudio.com

References