Decoding Muse Spark 1.2's 82.9%: One Benchmark Name, Three Different Scales

The headline figure quoted for Muse Spark 1.2 benchmarks is 82.9% on Terminal-Bench 2.1, and it appears in nearly every article about the launch sitting next to a rival's score as though both were measured with the same ruler. They weren't — three organisations publish Terminal-Bench 2.1 results on three incompatible scales. Understanding that is the most useful thing you can learn about this model's numbers, and about coding benchmarks generally. We applied the same lens to the agent in our Muse Code benchmark analysis.

How far apart are the scales? The official leaderboard tops out near 83.8%, Meta's own chart shows a top score of 86.7%, and Artificial Analysis' implementation runs near 89.5%. Comparing a number from one to a number from another is meaningless, and quoting either alongside a competitor's figure from a different source is worse than meaningless.

What Terminal-Bench actually measures

Terminal-Bench sets a model loose in a real terminal with a real task — fix this failing build, migrate this configuration, debug this service — and checks whether the end state is correct. It's a good benchmark precisely because it's realistic.

But that realism is also why the score is not a property of the model. Completing a terminal task requires an agent harness: something that decides which commands to run, how to interpret output, when to retry, how many turns to allow, and when to stop. A capable model in a poor harness scores badly. A modest model in an excellent harness scores surprisingly well.

A Terminal-Bench row is therefore always three things fused together — model + harness + effort setting — and every published number carries all three, whether the article quoting it says so or not.

What each source actually reports

The official board (tbench.ai). Its top rows are Claude Code with Claude Fable 5 at 83.8% ± 1.2% (xhigh, $552.67 to run), Codex with GPT-5.5 at 83.1% ($2,059.19), and Terminus 2 with Claude Fable 5 at 80.4%. Each entry names its harness and its cost, and submissions are verified.

Muse Spark 1.2 does not appear on it. Neither does Muse Code. The only Muse entry is mini-SWE-agent with Muse Spark 1.1 at 76.2% ± 1.2%, xhigh, submitted by Princeton on July 9, 2026, at rank 8 with a run cost of $198.05.

Meta's chart. Meta reports 82.9% for its own model, with Claude Opus 5 shown ahead at 86.7%. The methodology is pass@1 over five attempts, with each competing model paired to its own agent product — and Meta's own note concedes its harness may not be tuned for third-party systems. Nothing here has been externally reproduced.

Artificial Analysis runs its own Terminal-Bench v2.1 implementation, on which top scores land near 89.5%. It measured Muse Spark 1.2 moving from 78% to 80% between versions — a two-point gain on its scale, where Meta claims 6.7 on its own.

Same benchmark name. Three ecosystems. No conversion factor between them.

The arithmetic worth doing yourself

Meta claims 82.9%, and states that's 6.7 points above the previous generation.

82.9 − 6.7 = 76.2.

That is precisely the official board's figure for Muse Spark 1.1 running under mini-SWE-agent, a deliberately minimal scaffold built by Princeton researchers as a research baseline rather than a production agent.

Meta never published the harness behind its own baseline, so this is an inference, not a proven claim — and it should be stated that way. But if that row is the baseline, then the advertised "+6.7 generational improvement" is comparing a bare research scaffold to Meta's own purpose-built, co-trained agent. Some of that gain belongs to the model. Some belongs to the harness. The published number doesn't separate them, and nothing in Meta's materials lets a reader do so.

That's not an accusation of dishonesty. It's the structural problem with every vendor-run agent benchmark, and it's why the field needs verified third-party submissions.

The independent check

Vals AI runs every model through one common harness, which is exactly the control that would resolve this.

Under those conditions, Muse Spark 1.2 ranks #14 of 50 on Terminal-Bench 2.1.

Not second. Fourteenth. And the same evaluation places it #9 of 79 on SWE-bench — a solid coding result — while ranking it #1 of 44 on Finance Agent (v2), #1 of 136 on TaxEval v2 and #1 of 31 on Harvey's Legal Agent Benchmark, and 5th of 45 overall on the Vals Index at 71.88% for $0.69 per test.

The picture that emerges once the harness is held constant is coherent: a very good general model with unusual strength in long-horizon, document-heavy professional work, and competent-but-not-leading coding performance. That is a different model from the one the 82.9% implies.

How to read any coding benchmark from now on

• Find the harness. If it isn't named, the number isn't interpretable. "Model X scored Y" is an incomplete sentence for agentic benchmarks.

• Find the effort setting. Nearly every headline figure is maximum effort. Your production default probably isn't.

• Check who ran it. Vendor-run, third-party common harness, and verified leaderboard submissions are three different tiers of evidence.

• Never compare across sources. 82.9% from Meta's chart and 83.8% from the official board are not four-tenths of a point apart. They're on different scales.

• Check the cost column where one exists. The official board publishes run costs precisely because a score achieved for $2,059 and one achieved for $198 are not the same result.

And then, having done all that, run the benchmark that actually matters: your own tasks, your own harness, your own effort setting. That's the only comparison whose result transfers to your production system — and putting candidate models behind one OpenAI-compatible key makes it a day's work rather than a project. OrcaRouter carries Muse Spark 1.2 alongside 200-plus other models at 0% markup, which is enough to run the comparison honestly before anyone commits.

The takeaway

82.9% is a real measurement of a real thing — Meta's model inside Meta's harness at maximum effort, compared against rivals inside their own harnesses, by the company selling it. It is not a model score, it is not comparable to the official leaderboard's 83.8%, and the 6.7-point generational gain it advertises may fold a harness change in with a model change. The one apples-to-apples reading available puts the same model fourteenth of 50 at coding, and first in class at three professional domains nobody was marketing it for. Read the harness before you read the number.

Sourcing note: the 82.9%, 86.7% and +6.7 figures are Meta's own vendor-run results, unreproduced by third parties. The 83.8%, 83.1%, 80.4% and 76.2% rows are from the official Terminal-Bench 2.1 leaderboard. The 78% → 80% movement and the ~89.5% scale reference are from Artificial Analysis' own implementation. Common-harness ranks are from Vals AI. The 82.9 − 6.7 = 76.2 observation is our inference, not a Meta statement. Checked August 7, 2026.