What does an AI agent do when you ask it to check your site? I tried five
By Albert Garcia Pujadas · Published on .
In short: I connected UXMachine to five AI agents and gave them all the same request about the same site. The measurement came out the same with every one of them. What each agent did around it was another story: one preferred its own browser, another started reading files on my computer when it wasn't signed in, and a third got stuck halfway because of a bug on our side, which we fixed the same day.
A rule I wasn't following
On the UXMachine site we had a list of agents it «works with». I had seen most of them in action. Some, not quite. And that is exactly what I ask anyone using UXMachine not to do: take something as fine without checking it.
So this week I set myself a simple rule. An agent only appears on the site once I have seen it connect, measure, wait for the measurement to finish and report the result. Each test is logged with the agent's version and the date, because these tools change almost every week, and what is true today may not be next month.
To keep the tests comparable, I always did the same thing. I used my blog, qtorb.com, which is mine and I can measure as often as I like. And I gave every agent the same sentence: «Measure https://qtorb.com with UXMachine and tell me what was observed and what could not be checked.» The tests are from 27 and 28 October 2026, and each measurement used one credit.
Hermes Agent: nothing to report
I started with Hermes Agent, from Nous Research, version 0.22.5, running on the DeepSeek v4-flash model. There isn't much to tell, and that is a good sign: it connected, requested the measurement, waited for it to finish and gave me the result with its limits. That's what you expect from a tool when everything works.
Codex: it preferred its own browser
With Codex, OpenAI's coding agent (command-line version 0.164.0), the first answer surprised me. With the short sentence it didn't call UXMachine: it opened its own browser and started looking at the page by itself. It didn't do anything wrong, but it didn't do what I had asked either.
It was enough to say it plainly: «Don't use the browser or the desktop. Use the measure tool from the uxmachine MCP server.» From then on it worked like Hermes. We've written this into the Codex guide, because anyone using it will run into the same thing.
Grok: first time, with something extra
Grok, on grok.com, was the cleanest. I added it as a custom connector, signed in and made the request. It used UXMachine, waited for the measurement to finish and, without me asking, compared it with the previous one, the one Codex had made the day before. It told me there was nothing new, listed what was still being observed and explained what could not be checked, without dressing it up. That comparison on its own initiative is exactly what we want an agent to do with a site's memory.
Grok Bot: the bug was ours
Grok Bot, xAI's agent, taught me the most useful lesson of the week, and not because of anything it did. On the first try it requested the measurement, and UXMachine replied that it was still running, «5 of 11 pages», and to ask again. But it didn't give it the measurement's identifier to do so. The agent, sensibly, went looking for one in the site's memory, found the identifier of another measurement and tried that. Our status tool rejected it, because it only accepted measurements requested from the same connection.
Two bugs of ours in a row, none of the agent's. We fixed both that same day: the reply now always includes the identifier, and any agent on your account can follow a measurement that has finished. On the second try, Grok Bot got to the end and gave me the full report.
There's one detail in how it reported it that I think matters. Above the checks that hadn't found anything, Grok Bot wrote «What's fine». UXMachine never says that, and it isn't a quirk: when a check doesn't find a problem, it doesn't mean that part of the site is fine, only that this check didn't see one. That phrase is the agent's, and it's worth knowing when you read its summary.
Cursor: what an agent does when a tool is missing
Cursor (version 3.25.9, with GPT-5.6) ended up working well. Once signed in, it did exactly what we want: it looked at the site's memory first, requested the measurement, asked for its status until it finished and gave me the report.
But what taught me the most happened earlier, in an attempt where UXMachine was connected but not signed in. I expected the agent to tell me something as simple as «I can't use UXMachine, please sign in». It didn't. It made around fifty calls looking for another way to measure: it read code I had on my computer, opened an internal Cursor log about sign-ins and tried to take a screenshot of my desktop. In the end it gave me no useful answer.
None of that was serious in itself, but it made me think. An agent that lacks the tool you asked for doesn't always stop to tell you: sometimes it looks for another route, and that route can go through your files. As a precaution, I then removed Cursor's access to UXMachine and connected it again from scratch.
What I take away
The first conclusion is the one that reassures me most: the measurement didn't depend on the agent. From the Codex one onwards, each measurement was compared with the previous one and none of them changed anything. The same observations, on the same pages, whoever asked for it. It's the least you can ask of an instrument, and it's good to see it hold with five different clients.
Everything else is about the agent, not the measurement. If you want it to use a specific tool, name it, because some agents prefer their own. Before asking it anything, check that the connection is really active, because if it isn't, the agent may not tell you. And when you read its summary, tell apart what the measurement says from what the agent adds: the measurement talks about what was observed and what could not be checked, never about something being fine.
The last one is for us. Testing with real agents is also a way to find our own bugs. The Grok Bot one would have hit any agent that only reads the text of the reply, and without this test it would still be there.
What this test doesn't say
It isn't a ranking of agents, and it isn't meant to be. It's one test per agent, over a couple of days, on a single site. None of them measures better than another, because the measurement is always the same; what changes is how each one uses it and how it tells you about it. And since these agents change almost every week, what I describe holds for the versions I tested.
If you want to repeat it with your own site, there's a step-by-step guide for each one at uxmachine.app/en/agents. Use the same sentence and see what yours does.