Why doesn't UXMachine use AI to check AI-built websites?

It's one of the questions I get asked most, and I understand why. UXMachine is a website checker for AI coding agents: you use it to check a site after an AI agent has changed it. Yet it doesn't use any AI model to do that. It opens each page in a real browser, at desktop and mobile sizes, and applies a fixed set of rules. Measure the same page twice and you get the same answer twice.

That decision has a cost, and I'd rather be clear about it from the start: a general-purpose model finds more things than we do.

From five agents to none

The first version of UXMachine, in April, went the other way. Five AI agents read each page and debated among themselves until they reached a verdict. Over the summer I rebuilt the engine, but it still handed the interpretation of what it saw to a model.

On 6 August I opened a new repository and started from scratch. One of the first commits that day says it plainly: "no free model text". Since then, UXMachine doesn't consult any model when it checks a website.

Why I took AI out of the check

The model wasn't always wrong. The problem was that we didn't know when it was.

Looking at how the page was prepared before it was handed over, we found seven places where content was cut without anyone having documented why. The report looked complete, but there was no way to tell what the model had read and what had been lost along the way.

There was also a deeper reason. I haven't measured it, but it was the one that weighed most. What I wanted was to compare: measure before the agent touches the site, measure after, and see what changed. If the checker is a model, any difference between the two measurements could come from the site or from the model itself, which read it differently this time. And from the outside there's no way to tell one from the other.

What a model sees and we don't

In one of the first tests in August, a free model spotted an element on a page that worked as a button without being marked as one. We didn't catch it, because we had no rule for that case.

It happens more than I'd like. We only check what we have written down as a rule, and the list is public: these are our checks. If something isn't there, we don't look at it.

In exchange, every observation says which page and which width it was seen on, what state it should have and how to check it again. And when something can't be checked, the report says so: it doesn't leave it blank or count it as fine.

That's what makes comparing possible. If something stops showing up between one measurement and the next, the cause is in the page, not in a model that saw it differently that day. Even so, something no longer showing up doesn't mean it's fixed. It means we didn't see it this time.

Without AI there are errors too

We found that out a few days ago. On 13 October we measured 20 websites twice in a row, without touching anything in between. In two of them, changes showed up that didn't exist.

The error was ours. We identified each page by its path without taking the domain into account, so the home page of a subdomain and the home page of the main site counted as the same page. We fixed it on 18 October. I haven't repeated the test yet, so I can't say the problem is gone.

What I take from it: we found it, and we knew exactly which line of code it was in. With a model in the middle, I'm not sure we would have found it.

Where I do use AI

In almost everything else. The websites we check are changed by AI agents. Those same agents read our measurements through the connector. And I build UXMachine with agents, as I explained in this article (in Spanish).

The one thing I don't do is ask one AI to sign off on another AI's work.

What this note doesn't show

It doesn't show that our rules cover what matters most to the people who use a website. We measure what the page shows when it loads. We don't give opinions on design, copy or brand, and none of this replaces testing with people.

Nor am I saying models are no good for checking websites. They are, and sometimes they see more than we do. But they're no good for what I need, which is to compare.

Data and sources

• First version, April to May 2026: five agents and a model. Second, June to September: a separate engine that still called a model. Third, since 6 August: no model calls.
• Noise test on 13 October 2026: 20 websites, each measured twice through the same path the product uses. In two, changes appeared that were caused by our own instrument. Fixed on 18 October; the test hasn't been repeated yet.
• Each measurement checks up to 11 pages, on a computer (1280×800) and on a phone (390×844).
• Previous note: how to check a website after AI agent changes.

← All notes