Researchers at EPFL and Apple took the machinery around AI agents apart, one layer at a time. Search trees. Planners. Teams of agents that talk to each other. Almost none of it helped. One thing helped more than everything else together: giving the model a computer to work on.
Their paper is about AI agents that do machine learning work, not websites. But the lesson carries over to anyone who picks an AI website builder. The number of agents in the product matters less than you think. What the agent can see and do matters more.
What the study tested
The team held three things fixed: the AI model, the hardware and the time budget. Then they climbed a ladder of designs, from simple to elaborate (Brilliantov, Hernández-Cano and Abbé, arXiv 2609.40303):
- Chat. The model writes a whole script in one reply. If the script fails, the model gets the error back.
- One session with a computer. The model can read files, write files and run code. It explores, tests and fixes its own work.
- Add-ons. Search over many attempts, subagents, three agents in parallel, agents that message each other, scheduled reminders and strategy guides.
The result was clear. In the authors' words: "Giving the model a shell and a filesystem instead of a chat interface is the single largest effect we measure."
Everything after that step barely moved the result. One plain agent session matched or beat four published multi-agent systems on every strong model they tested. With the GLM 5.2 model, the plain session won a medal on 62.5% of the tasks. The best of the elaborate systems won on 47.1%.
Some add-ons made things worse:
| Change to one plain agent session | Medal rate |
|---|---|
| No add-ons | 55.7% |
| Subagents to hand work to | 45.2% (not a significant change) |
| Three agents in parallel | 52.4% |
| Three agents in parallel that message each other | 33.3% |
Source: Table 1 in the paper, 14 tasks, 24 hours each.
Their conclusion: "The leverage is in the model and in the runtime it is given, not in the scaffolding built around them."
For a website, the environment is the product
In the study, "a computer to work on" meant files, a terminal and the power to run code. For a website agent, it means three things:
- The real files. The pages, styles and content you publish, not a copy pasted into a chat.
- The result. The page as a browser renders it, after the edit, at the width your visitors use.
- The context. Your product notes, release notes and brand files. These are the facts a model cannot know from its training.
Now look at where most AI website agents live. In a browser builder, the agent runs on the vendor's servers. Your site is rows in the vendor's database. The agent can use only the tools the vendor builds and pays to run. So when the vendor wants "more AI", the easy move is to add layers on top: more agents, more workflows, more credits. That is the scaffolding layer, and it is the layer the study found added the least.
What an agent on your device can see
Splitleaf is a desktop app. Your site is a folder of files on your computer, and the agent works in that folder. This is what it sees today:
- Every file you publish. Pages, components, styles and CMS content are ordinary files. The agent reads and edits the same files the Designer shows you.
- The page as it renders. After an edit, the agent can open the page in a real browser at any screen width. It can take a screenshot and read the final style of any element. It does not have to guess if the button moved.
- A check on every edit. Each file the agent writes is checked for broken HTML and CSS. When the agent measured a page with a screenshot and then edits it, the app measures again. If the result changed, the agent is told.
- Your folders, where they are. Link a folder with product docs or release notes, and the agent reads it in place. No upload, and no copy that goes out of date. More in Agentic CMS: the context problem nobody solves.
- The model you choose. The study found the model to be the main driver. You connect your own key from Anthropic, OpenAI or OpenRouter, or you work with Claude Code.
The agent stages every change. You keep it or undo it in the Designer. Nothing goes live without you.
We learned to stop managing the agent
The study tested reminders that report progress, such as how much time is left. They lowered the score. We saw the same thing in our own app. Our agent used to get a warning when its working memory, the context window, was 70% full. The models took that as an exit. With V2 we help by summarizing the context.
Going forward Splitleaf is focused on one agent per request. We do not build a planner, a reviewer agent or a team of agents that talk to each other. We put the work into what the agent can see, not into micromanaging it.
Five questions to ask an AI website builder
- After an edit, what can the agent see? The rendered page, or only the text it wrote?
- Does it work on the real files? Or on a copy, a snippet or a database it reaches through an API?
- Can it read my product context where it lives? Or do I upload it again every time it changes?
- Can I choose the model? The model matters more than the features around it.
- Is "more agents" measured? If a tool sells a team of agents, ask what it improved, and against what.
See how the agent works in Splitleaf on the build with agents page.