Many, and eventually most, consumers will use AI agents to complete transactional tasks, and in 2027 this will quickly cannibalise transactional organic traffic in much the same way that AI Overviews (AIOs) have cannibalised informational search traffic.
Every organisation needs to decide how much functionality they wish to make available to these agents:
- ✅ Completely allow them, and risk being disintermediated.
- ⚖️ Allow agents to do some tasks, and hope your competitors don’t allow more.
- 🚫 Completely disallow them, and risk losing those customers.
We are 100% going to have another terminology war over what this is called, similar to GEO vs. AEO vs. AI Search Optimisation. But I’m going to call it “Agent Experience”.
So, assuming you as a brand want to optimise for agent experience, how actually do you do this?
1. CDNs
CDN bot protection is the biggest barrier to agent experience at present. This can be deliberate, but is often accidental, as Cloudflare or CloudFront WAF rules detect a suspicious request and apply a challenge, or even just return a 403.
In October, Cloudflare’s AI bot directory added Meta Muse, which it blocks by default for ad-supported sites.
Brands need to define an organisational policy and WAF rules for how they treat different categories of requests:
- User-initiated agent requests (e.g. ChatGPT-User)
- Model-training bots (e.g. GPTBot)
- Autonomous agent requests (e.g. Muse, Dot, Grok Bot)
2. WebMCP
WebMCP (Web Model Context Protocol) is a proposed browser standard that Google is helping develop. It provides JavaScript APIs and HTML form annotations so that agents know how to interact with page features. It supports ‘declarative tools’, defined in HTML, and ‘imperative tools’, defined in JavaScript.
Some of Google’s agent-readiness guidance covers existing best practices that also happen to be useful for agents, but WebMCP itself requires explicit adoption.
Google’s Lighthouse agentic browsing checks
2.1 Imperative API
Define tools using JavaScript, covering actions such as form input, navigation and state management.
For example, use registerTool to add a single tool to the model context:
await document.modelContext.registerTool({
name: 'toggle_layer',
description: 'Control pizza layers (sauce, cheese). Use "add", "remove", or "toggle".',
inputSchema: {
type: 'object',
properties: {
layer: { type: 'string', enum: ['sauce-layer', 'cheese-layer'] },
action: { type: 'string', enum: ['add', 'remove', 'toggle'] },
},
required: ['layer'],
},
execute: async ({ layer, action }) => {
await toggleLayer(layer, action);
return `Performed ${action || 'toggle'} on layer: ${layer}`;
},
});2.2 Declarative API
Add annotations to standard HTML forms to expose them as WebMCP tools.
Start with a sound accessibility tree: prioritise semantic HTML and proper labelling. This is accessibility good practice, regardless of WebMCP:
<form action="/candidates" method="get">
<label for="skills">Skills</label>
<input id="skills" name="skills" type="text">
<label for="location">Location</label>
<select id="location" name="location">
<option value="">Any location</option>
<option value="london">London</option>
<option value="manchester">Manchester</option>
</select>
<button type="submit">Search candidates</button>
</form>Then add WebMCP attributes to the opening form tag to expose it as a named tool:
<form
action="/candidates"
method="get"
toolname="searchCandidates"
tooldescription="Find candidates by skills and location."
>3. Register a connector
Meta Muse uses “connectors” to enable integrations.
It has a community programme that brands can use to submit their own connector. You can see third-party lists of fairly major brands that have already created their connectors.
Meta’s connector documentation
4. Adopt an industry-specific protocol
At present there are only two industry-specific protocols I know of, both competing for ecommerce:
UCP will almost certainly become the de facto standard for ecommerce, and only involves putting a JSON file in the .well-known directory. ACP is a larger-scope implementation.
I expect to see standards for other industries emerge.
5. Fix Cumulative Layout Shift (CLS)
CLS, long one of the Core Web Vitals, measures visual stability. Reducing layout shift is important for agents that rely on element positioning to interact with a page.
6. llms.txt
Despite some teams at Google talking down the value of llms.txt files, the Chrome Lighthouse agentic readiness report checks for the presence of a machine-readable summary at the domain root.
It is true that no major AI lab currently uses llms.txt, but given that AGENTS.md has such a dramatic benefit for the performance of coding agents, it seems inevitable that a standard like this will emerge.
Measurement
At present, measurement is nearly impossible at scale, but tools like Profound are actively working on it, much as they already do for informational generative AI.
Agent traffic is very hard, though not impossible, to distinguish from regular user traffic. This will improve as protocols like the Personal Agent Protocol become available.
There will, however, be an ongoing tension between agent providers like Meta, Google, xAI, OpenAI and Anthropic, who deliver value to their users, and brands like Amazon that explicitly want to keep these agents out.
The hardest challenge of all, though, will be for brands to identify if and how much functionality they want to enable. This will be easy for organisations that provide an offline service, but for intermediaries like aggregators - it will be a fine line between losing market share and being disintermediated.
