Optimizely published its marketing AI benchmark twice, with two different scores

Optimizely held Opticon NYC at the Javits Center on 1 September 2026 and announced Mark AI, a set of models it describes as purpose-built for marketing, alongside a benchmark for measuring marketing AI that it describes as open source. The benchmark names Claude Code as its comparator. Optimizely described the same comparison in two of its own documents, published a day apart, and the two report different results.

What Optimizely announced

Optimizely’s field-notes recap of the event lists five Virtual Teammates available now, covering Chief of Staff, SEO and AI Search Analyst, Marketing Analyst, Personalization Strategist and CRO Manager, plus Agent Visibility, built with Conductor, and Agent-Ready Experiences delivered over WebMCP. The same recap carries Optimizely’s claim that AI referral traffic and crawlers now account for 60% of web visits.

The benchmark, and what it tests

Optimizely’s press release calls it Mark-Bench and describes it as “an open source benchmark built to evaluate AI performance for the marketing domain specifically”. On scope, the release says Mark-Bench “tests models directly against 285 tasks spanning 15 marketing functions, and over 6,000 criteria including writing a press release, creating a social post, and writing email copy”. Optimizely’s field-notes recap, published the following day, calls the same announcement the AI Benchmark for Marketing, and describes Mark AI’s models as “evaluated across 285 marketing functions”, which restates the release’s 285 tasks across 15 functions as 285 functions.

67% in one document, 58% in the other

The press release of 1 September 2026 states: “As evaluated by Mark-Bench’s all-pass rate on default configurations, Optimizely Agent Platform scored 67% compared to 60% for Claude Code at 2x lower cost.”

The field-notes recap of 2 September 2026 states: “Optimizely’s Agent Platform scored 58% all-pass against Claude Code’s 52%, with 2.1 times the efficiency.”

The release’s two scores sit 9 and 8 points above the recap’s. The reported margin is 7 points in the release and 6 points in the recap. The cost claim is worded as “2x lower cost” in one document and as “2.1 times the efficiency” in the other. Optimizely’s published material does not specify what differed between the two accounts, and the two were published a day apart by the same company about the same product.

Who scored it

The benchmark was written and run by the vendor whose product it ranks first, and every figure above is Optimizely scoring Optimizely.

What the scores describe

The tasks Optimizely names are marketing deliverables: a press release, a social post, email copy, graded on an all-pass rate across the criteria the benchmark defines. Site building sits outside that scope, and so do content modelling, template work and moving a CMS estate. Nothing in the two documents describes a test covering any of it, and WAYF has run no comparison of its own, so this post reports what Optimizely published and goes no further. Our account of directing a coding agent through a site rebuild is in replacing Webflow with Claude Code.

Optimizely is one of the platforms teams ask us to move off, and the route is set out in the Optimizely to Payload migration guide. The wider pattern of platform vendors shipping agents into their own products, and the questions that leaves open for a team choosing a platform, is covered in no-code platforms ship their own agents.

For a marketing team weighing the announcement, the checkable fact is the one Optimizely supplied: the same product, on the same benchmark, against the same comparator, was reported at 67% and at 58% in two days.

FAQ

1. What is Mark-Bench?

   Mark-Bench is the name Optimizely's 1 September 2026 press release gives to the marketing AI benchmark it announced at Opticon NYC, described there as an open source benchmark built to evaluate AI performance for the marketing domain specifically. The same release says it tests models directly against 285 tasks spanning 15 marketing functions, and over 6,000 criteria including writing a press release, creating a social post, and writing email copy. Optimizely's field-notes recap of the event, published 2 September, calls the same announcement the AI Benchmark for Marketing.

2. What scores did Optimizely report against Claude Code?

   Two pairs, in two documents. The press release of 1 September 2026 states that, as evaluated by Mark-Bench's all-pass rate on default configurations, Optimizely Agent Platform scored 67% compared to 60% for Claude Code at 2x lower cost. The field-notes recap of 2 September 2026 states that Optimizely's Agent Platform scored 58% all-pass against Claude Code's 52%, with 2.1 times the efficiency. Neither document says what differed between the two accounts.

3. Who ran the benchmark?

   Optimizely. The benchmark was authored and published by the vendor whose product it ranks first, and the scores in both documents are Optimizely's own results on Optimizely's own test. WAYF has not run the benchmark and has not reviewed its methodology beyond what the two published documents state.

4. Does the benchmark say anything about coding agents on a website project?

   It measures a different job. The tasks Optimizely names are marketing work, graded on an all-pass rate: press releases, social posts, email copy. Site building, CMS modelling and migration work are not in the described scope, so a result there would have to come from a test scoped to that work.

Sources

• Optimizely, Optimizely introduces purpose-built AI models for marketing (1 September 2026)
• Optimizely, Postcards from Camp Opticon NYC (2 September 2026)