Sanity Content Agent vs. Payload: An Open-Source Alternative, Tested on 31 Real Editorial Tasks

A couple of months ago I built an MVP of a content agent for Payload and wrote about it in a separate article. There I put it next to Sanity’s agent on simple tasks: finding links, verifying a list of articles, and running a bulk replacement. The results were very close, and I promised to come back with a more honest test - not on tasks I made up, but on the ones editors ask for every day.

After that article, we handed the agent to two of our content managers as an internal beta. For five weeks, they used it as part of their normal work. I refined the agent based on their requests and deliberately did not coach them on how to phrase things “correctly”. From those requests, I picked 31 tasks and ran them through both agents - ours and Sanity Content Agent - word for word, exactly as the editors wrote them. This article is the result: a table with the time and outcome for every task, plus a closer look at cases where the agents’ differences turned out to be differences between the CMSs themselves.

Who this article is for #

If you are choosing a headless CMS and see Sanity as the platform that already ships an AI agent, but you are also looking at Payload as the more flexible, independent alternative - this is a chance to compare how the two agents actually handle the same tasks, not what the decks promise. Good news if you lean towards Payload: a content agent is coming there too. We are working on taking ours out of internal beta. And if you simply want to know what content agents inside a CMS can do today, skip to the table: every task in it comes from real editorial work.

One important detail about the setup. Both agents worked on identical documents. The content of focusreactive.com - 148 articles and 123 landing pages - exists in two copies, one in Payload and one in Sanity, with the same texts, blocks and references between documents. Each agent received its task inside its own CMS, on its own copy. Payload Content Agent ran on OpenAI models in the same configuration we use in production: the orchestrator runs on gpt-5.4 - it reads the request, builds a plan and hands out work; the sub-agents that edit text and rich text, and the reviewer that checks the result, run on gpt-5.4-mini; background memory tasks run on gpt-5.4-nano. That is our configuration, not the only one possible. The plugin supports multiple providers out of the box - OpenAI and Anthropic - so you bring your own API key and pick the model for each role based on budget and quality. Which model runs inside Sanity Content Agent, the platform does not disclose: it picks the model itself, and the cost is folded into the credit price.

TL;DR: AI Content Agent Battle #

Two content managers worked with the agent for five weeks. I consolidated their work into 30 sessions: 31 documents changed, roughly $20 in model cost.

From their requests, we’ve assembled 31 tasks and gave them to both agents word for word. Payload Content Agent was faster on 20 tasks, Sanity Content Agent on 11; median time 2:29 vs. 3:02.

Each does something the other cannot. Sanity resizes images and generates new ones. Payload Content Agent runs as a plugin inside the Payload admin and connects to external tools over MCP: in our case Ahrefs, so the agent answers SEO questions with real search volumes right in the chat. That makes it more than a chat for edits - it is an editor’s workbench inside Payload that you can wire up to any service your team uses.

The heaviest task - appending a paragraph to each of 148 articles - went to Sanity: one operation across the whole dataset versus 18 minutes of document-by-document work on our side. Landing page clones, reference handling and assembling pages from blocks went to Payload, faster and more precisely.

The 31 tasks cost $12.50 on the Payload agent, about $0.39 per task. Sanity’s billing only shows a total for all runs, including repeats, so we do not derive a per-task price. Both agents write drafts only. A human always publishes.

Five Weeks of Pilot #

The two editors work in different styles on different kinds of tasks, and together they give a complete picture of what a content team actually does. That shaped the task set more than any test of mine could.

The first one runs the blog. His requests are short and practical: “change all Jamstack to JAMstack”, “check the grammar”, “write a meta description for the article”. He rarely opens the document before writing to the agent and expects it to figure out which article he means.

The second one builds landing pages. His language is different: “take the Official Partner block from page X and put it on Y”, “replace the case studies and testimonials with the ones from the Framer page”, “make a copy of this landing, but about Strapi”. He thinks in blocks and references between documents, and almost every task of his is assembling a page from ready-made parts.

Five weeks in numbers:

Three things I noticed over those five weeks shaped how this comparison turned out.

1. Firstly, the agent’s work is not only search and audit, as I assumed when I built the MVP. The bulk of the load was page production. Those requests were only a quarter of the total, but they took four-fifths of the agent’s work. Clone a landing page, move a block, swap references - each of those costs as much as a dozen text edits. That is why the set has eight landing page tasks, and they are the heaviest of all.

2. Secondly, the most expensive mistake an agent can make is not a slow answer or an imprecise edit. It is a confident “done” over a document where nothing changed. That happened during the pilot, and I turned every such case into its own task in the set: the result of each task is verified against the document, not against what the agent says in the chat.

3. Thirdly, I took the editors’ requests as they were - their wording, their typos, their gaps - and gave them to both agents in that form. An agent should understand people in their everyday phrasing. But prompt engineering still matters, and it is worth keeping in mind: the more precisely an editor names the document, the field and the desired result, the fewer clarifying questions there are and the faster and more predictable the answer is, for both agents. You can see it in the table in the tasks where the first request was too broad and a second turn was needed.

How the runs worked. Sanity Content Agent ran in the Studio side panel, Payload Content Agent in the chat inside the Payload admin. Time was measured from sending the prompt to the agent’s final answer.

In Short #

Two tasks neither agent solved. Neither could verify data on a site that blocks bots. And neither CMS gives the agent an editor’s activity history as a log: Sanity shows the content where that person is listed as the author instead.

The Comparison Table #

Tasks are grouped by type of work, from simple to complex. Time is minutes:seconds to the agent’s final answer; the faster one is in bold.

Reading and Questions About Content #

SEO Fields #

Edits Inside a Single Article #

Grammar #

Links #

Bulk Operations Across the Blog #

External Data #

Media #

Landing Pages: Blocks #

Landing Pages: Rewrite #

Landing Pages: Create #

What This Shows #

If you only look at the bottom line, the picture is even: Payload is faster on twenty tasks out of thirty-one, but the medians nearly match - 2:24 vs. 2:37. Two other observations seem more interesting to me. First: over the course of the runs, it felt like Sanity was getting faster at the same requests with each repeat, as if learning from them. I did not measure this separately, so I offer it as an observation, not a fact. Second: each agent has its own strong tasks. Sanity wins the two heaviest - a bulk write across a hundred and fifty articles and checking a whole page against its topic. Payload wins landing page clones, reference handling and anything that requires stepping outside the CMS.

Below are five stories where this is clearer than in the table. They also explain why the winner in each of them is not so much the agent as the CMS architecture underneath it.

Task 17. A Paragraph at the End of Every Article #

In the first article, neither agent finished this task: Sanity reported it done without changing the documents, and our agent stopped on the volume. A month and a half later, the picture is different.

Add this paragraph as the last paragraph of the body of every published blog article: “Editor’s note: …” Link the words “CMS migration guide” to https://focusreactive.com/blog/cms-migration/. Change nothing else.

Sanity did the task completely: one operation, the whole blog. The only skipped post is the guide article the link points to: the agent decided on its own that linking to itself would be odd. Our agent completed the task in eighteen minutes. It no longer stops on volume, but it takes every article through the full cycle - open, read, decide, write, verify - and a language model runs at every step. Multiply that cycle by a hundred and fifty articles and you get eighteen minutes instead of eleven.

This is the exact bottleneck I wrote about in the first article, before I knew what it looked like in numbers: bulk operations are not solved by how smart the model is, but by how the work is organised. For Sanity it is one mechanical mutation across every document. We do not have that mechanism yet, and it is the next step in development.

Field selection is a separate story. In articles migrated from Sanity to Payload, the text lives in a markdown field, while a field with the telling name content holds a single heading, and the agent has to work out which of the two the site actually renders. Sanity has no such fork: an article has one body. This is the flip side of Payload’s flexibility: the schema can be anything, and the agent has to understand it rather than trust field names.

Task 29. A 1:1 Landing Page Clone for a New Topic #

This is the main task of the pilot: landing page clones accounted for a quarter of the requests and four fifths of the agent’s work. If your editors produce pages in series, look here first.

Take this landing /sanity-expert-agency/ as a base and make a copy with exactly the same look, but on the topic of hygraph-expert-agency. Fill it with the matching content, blocks and links.

Both did well here, and the differences are in character. Sanity also draws an image and finishes in seven minutes. Payload is a minute and a half faster and more meticulous: not a single word of the old topic remains, and every link points to a live page. One thing was left to the editor with both: the projects slider kept the old projects under a new heading. Neither agent worked out that “the same projects” and “the matching projects” are not the same thing.

Task 31. A Landing Page From Existing Blocks #

This task came from the pilot verbatim. The editor who builds landing pages asked: “build a new page from scratch out of our blocks”. The first version of the agent did not understand what “our” meant: it built blocks from empty schemas and saved nothing in ten minutes. After that I added a block library to the agent - a catalogue of block types with real examples and their references, which it now assembles pages from. It is a good example of how the pilot changed the agent: not according to my plan, but according to what editors asked for.

Make a new landing about headless CMS for e-learning from our existing blocks, with new texts. Slug: headless-cms-for-elearning

Both finished in three minutes, one second apart. Payload assembled ten sections. The intro, case studies and testimonials are references to real documents on the site, not invented text; all the copy is new and about e-learning. One addition is left to the editor: there is no hero section in the set, the page opens straight with the intro, and the header has to be added by hand.

Task 8. A Table Broken Three Ways #

A small task, but it shows well how each agent understands the boundaries of a request.

fix my markdown table in the Headless CMS Architecture vs. Traditional CMS

One table, the same three breakages, and both handled it cleanly: not a single cell rewritten. Sanity did exactly what was asked in four minutes. Payload fixed the table in a minute and a half and allowed itself one liberty: it wrote “Criteria” into the empty header cell. Whether that is appropriate is the editor’s call, but it is the only thing they will have to check.

Tasks 18 and 20. What Only One of Them Can Do #

There is nothing to compare here: these are two different views of where an agent ends. For Sanity, the agent is part of the Studio: content, media, generation, all inside the platform. Payload Content Agent is built differently: it is a plugin inside the Payload admin that connects to external tools over MCP. Today that is Ahrefs; tomorrow it could be analytics, an issue tracker or any other service the team works with. The connection is declared in the plugin config, with no separate integration per tool. The result is that the editor gets content edits and data from external systems in one window, and that is what makes the agent a strong working tool specifically inside Payload.

Why the Agents Perform Differently #

The safety principles I described in detail in the first article - drafts only, no deletions, an independent reviewer - did not change over the five weeks of the pilot. One thing did: the checks and the reviewer now run inside a single editing session instead of sending the agent around for a second pass. The agent sees exactly what did not add up and fixes only that, remembering what it has already done. For a team choosing a CMS, this means one thing: the agent publishes nothing on its own, and on both platforms the final word stays with a human.

What It Costs #

When choosing a platform, the agent’s price matters as much as its speed, but the two are billed differently: Sanity charges credits for the agent’s work, while the Payload agent pays the model provider for tokens. Both figures are real - from Sanity’s billing and from the OpenAI console for the agent’s key - but they cannot be compared directly. During the comparison I ran Sanity more than 32 times: some tasks were repeated while I was debugging the scenarios, and the billing shows only one total for everything. So we do not derive a per-task price for Sanity. On Payload every task ran once, and the cost is counted precisely. The billing model itself is an advantage of the Payload agent: you bring your own OpenAI or Anthropic key, pay the provider directly at its rates, see the spend per request in its console and can manage costs - switch to a cheaper model for edits, move up to a stronger one for planning, cap the budget at the key level. There is no platform fee or vendor margin in the price. Only tokens.

For scale: five weeks of the pilot with two editors cost about $20. What costs the agent the most is the same thing that takes it the longest: bulk writes, where the model takes part in every document.

The Takeaway: If You Are Choosing a CMS by Its Agent #

The agent does not replace the usual criteria for choosing a CMS: data model, hosting, price, engineering team. But it shows clearly where each platform is heading. In the agent, a platform has nowhere to hide: it can do exactly what its architecture allows.

Let me be upfront: I have no goal of outrunning Sanity’s agent on speed. Sanity was the first to show how a content agent should work inside a CMS, and its agent was my reference point and source of ideas when I built Payload Content Agent. Every measurement in this article serves one purpose: to understand where our agent already delivers at least the same effectiveness, where it does not yet, and what needs finishing to get there. The table is a working tool for development, not a scoreboard.

Sanity built its agent around a single content store and a single query language. One operation covers the whole dataset, releases give you a safe “undo everything”, and on bulk edits that advantage shows immediately. The agent is already built into the platform, with nothing to install or configure. If your editorial team works in large series of changes and does a lot with media, Sanity gives you all of that out of the box.

Payload gave its agent the same things it gives a developer: the schema, the Local API and the ability to connect external services. The agent sees the structure of a page down to every reference and can step outside the CMS - into Ahrefs, into analytics, into any service you connect. If your editors assemble pages from blocks and keep three other tools in their heads besides the CMS, this is your option. And it is not tied to our site: the plugin reads the schema of any Payload project and does not ask you to rework your content model for it.

If you are choosing between these two CMSs right now and the presence of an agent tips you towards Sanity, keep in mind that for Payload this difference is about to disappear. We are working on taking our agent out of internal beta and making it available to other teams. Choose the platform by how it handles content, hosting and development. The agent will be there either way.

Both agents write drafts only, and both make mistakes - the table above leaves no doubt about that. The difference is where those mistakes are visible: Sanity shows them before the write, Payload in the diff afterwards. Which of the two modes suits your team better will tell you more about the choice than a difference measured in seconds.

What It Changes for an Editorial Team #

Look at the task list again through a content manager’s eyes. Change the spelling of a term in a hundred articles. Append one paragraph to the end of every published article. Check that a link goes where it should. Fix a table, restore a missing space, write a meta description, bold one fragment in the middle of a long text. None of these tasks takes talent, but every one of them takes time. Maintaining a living content library is made of tasks like these, and for years they ate into an editor’s working day because there was nobody else to do them.

Over the five weeks of the pilot, that routine moved into the chat. The blog editor stopped opening every article to find one dash in it and writes the agent a single sentence instead. The landing page editor stopped assembling a page by hand from fifteen blocks and simply describes what the result should be: take this block from here, replace the case studies with the ones from there, make a copy about another topic. Five weeks of that work cost about $20 in model spend.

What the team gets out of it. Library maintenance stops being postponed: an edit that used to wait “until the next big content refactor” is now done the same day. The site lives in one state instead of three: older articles get the same links, the same terminology and the same formatting as new ones. Content errors are found and fixed before a reader or a search engine notices them.

But the point is not speed. The goal is not to build a tool that beats Sanity’s agent on the clock. It is to take the routine off the editor and do it reliably. The editor spends the freed-up time on what the agent will not do: deciding which topics the site needs, how to position the product, what a reader should take away from a page and why they would want to come back. The content manager stops being a CMS operator and becomes an editor in the original sense of the word: responsible for meaning, not mechanics.

For the business it is a simple trade: routine goes to the tool, strategy stays with the person, and each does what it is better at.

If you want to try both against your own content, let’s talk: we work with both platforms and will help you compare them on your tasks, not ours.