Claude Opus 5 Benchmarks: Every Number, Verified Against the Source

Opus 5 shipped July 24, 2026. I pulled the Claude Opus 5 benchmarks out of the actual system cards instead of the blog posts written about them, and found that two of the numbers everybody is quoting are attached to the wrong model.

The Claude Opus 5 benchmarks, verified

Source: Claude Opus 5 System Card, July 24 2026 — Table 8.1.A and §8.2

Anthropic's announcement page for Opus 5 does not print a single SWE-bench number. It headlines Frontier-Bench v0.1 and shows most results as images. The numbers below come out of the system card PDF itself.

Every Claude 5 model, side by side

Each score below is quoted from the system card of the model it belongs to. Read the source column before you compare any two rows.

"n/r" means not reported in that model's card. Sonnet 5's introductory price of $2 / $10 runs through August 31, 2026. Opus 5, Fable 5 and Sonnet 5 all carry a 1M-token context window and 128k max output.

The column everybody misreads

Here is where most of the wrong numbers on the internet come from, and it is almost funny once you see it.

In the Fable 5 system card, the summary table's first score column is not Fable 5. It is Mythos 5, the limited-release sibling that ships without the safety classifiers. Fable 5 is the second column. If you skim that table left to right the way you read everything else, you hand Fable a set of scores that belong to a different model.

Two Anthropic cards independently agree on 80 for Fable, because the Opus 5 card lists Fable's SWE-bench Pro in its own table too. The 80.3 floating around belongs to Mythos.

It gets worse on the other side of the fence. One widely-linked review publishes "SWE-Bench Verified 88.7%" for GPT-5.5 and cites OpenAI's launch announcement as the source. That announcement contains no SWE-bench Verified score at all. I checked the whole page. OpenAI reported GPT-5.5 at 58.6% on the public split of SWE-Bench Pro, with a footnote warning about memorization, and nothing else. The 88.7 was invented and given a real citation, which is worse than no citation.

Both coding benchmarks got retracted this year

This is the part nobody put in a headline, and it changes how you should read every table above.

Read that again. Inside five months, the two coding benchmarks the entire industry quoted at each other were both publicly disowned by one of the labs that quoted them. Not by critics. By OpenAI, about evals OpenAI itself had recommended.

The flaws were not cheating. They were spec defects: tests that enforce implementation details the prompt never asked for, so a functionally correct fix fails. Underspecified prompts. Low-coverage tests. Misleading task descriptions. In other words, a model could be right and score wrong.

The theory: what a benchmark score actually measures

The transferable idea here is older than any of these models, and once you have it you cannot read a leaderboard the same way.

Why it's true. This is Goodhart's Law with a compiler attached. Goodhart's formulation is that when a measure becomes a target, it stops being a good measure. The mechanism is that everyone optimizes the proxy rather than the thing the proxy was standing in for. Labs tune scaffolds, retry counts, effort settings and prompt formats against the eval, and the score climbs while the underlying capability moves less than the graph suggests. Meanwhile the eval itself ages: its tests were written against a snapshot of reality, and the parts of the task the test forgot to specify quietly become the parts models get punished for guessing.

Measurement people call the missing piece construct validity. The construct you care about is "can this thing fix my code." The measure is "did the repo's existing test suite go green." Those overlap. They are not the same thing, and the July audit is exactly what the gap looks like when someone finally checks: roughly a third of the tasks could not have measured the construct no matter which model ran them.

Here is the proof inside Anthropic's own documents. GDPval-AA is an Elo score, run externally by Artificial Analysis. Watch what happens across three cards released seven weeks apart:

Same models. Same publisher. Same externally-run leaderboard. Nearly 300 Elo points of difference, because they are different snapshots and different versions of the eval. Neither number is a lie. They are simply not comparable, and any table that stacks them in one column is fiction assembled out of true parts.

How you apply it. Three rules I now use before quoting any score. One: find the harness. If two numbers came from different scaffolds, they are two different experiments. GPT-5.5's 83.4 on Terminal-Bench was run on the Codex harness, Claude's on mini-SWE-agent, and putting them in adjacent cells implies a comparison that was never performed. Two: find the trial count. Opus 5's 96.0 is a mean over five trials; a single-run number next to it is a different kind of object. Three: read the footnotes before the table, because that is where the fallbacks, the effort settings and the memorization warnings live.

Why Fable 5's numbers are product scores, not model scores

I have been honest that Fable 5 never fully convinced me, and reading its card finally explained the feeling.

Fable ships with safety classifiers that can decline a request outright. When that happens mid-benchmark, the run falls back to Opus 4.8 for the rest of the trajectory. On Terminal-Bench 2.1, the card states that 20.9% of Fable's trials hit a safety refusal and finished on Opus 4.8. Fable's published 84.3 is therefore roughly four-fifths Fable and one-fifth the previous Opus.

Anthropic says so plainly in the card's own table note: Fable's scores reflect its production safeguards, including the fallback, which is why some benchmarks come in slightly lower for Fable than for Mythos. That is a fair and unusually transparent disclosure. It also means Fable's numbers describe a product, and Mythos's describe the model underneath it.

Which matches what it feels like to work with. Fable has the wider band. It reaches further on the problems it takes, and then some fraction of the time it simply declines and something else finishes the job. That inconsistency is the thing my gut was reacting to before I had a number for it.

How I actually test a model

So I stopped outsourcing the question. I keep a set of my own scripts where I already know the exact bug count, because I put the bugs there. Then I hand the whole thing over cold and watch.

Catching them is the easy half. Every frontier model finds most seeded bugs now, and a pass rate on its own tells me almost nothing. What I grade is how it fixes them, on three things:

That is a rubric, not a leaderboard, and it is deliberately unfair to whichever model I happen to be testing, because my scripts are not in anyone's training data and the bugs are the kind I actually write. Which is the whole point. The construct I care about is "can this thing fix my code," so I measure that directly instead of trusting a proxy someone else validated against someone else's repos.

You can build the same thing in an afternoon. Take three files you know cold. Break them in ways you have genuinely broken things before, write down the count, and keep the file somewhere the model cannot see until you hand it over. Run every new release against it. Grade the diffs, not the pass rate. Inside two releases you will trust your own set more than any table on the internet, including the one further up this page.

My verdict on Opus 5

I am still early in my own runs, so take this as a first read rather than a final grade.

This is the first model since the last Sonnet that I believe is a real upgrade rather than a repackaging. Everything between them felt like the same capability with different padding. Opus 5 does not. The fixes come back smaller and more targeted, and it argues with me less about things it turned out to be wrong about.

The benchmark that backs the feeling is ARC-AGI-3, because it is the one built to resist memorization. 1.5 to 30.2 from Opus 4.8 is not a scaffold improvement. Something underneath moved. Frontier-Bench points the same way, and it is a brand new eval, which means nobody has had time to overfit to it yet. That is a genuinely useful property for about six more months.

And it has made Fable easy to forget, which I did not expect to write. Fable still has the wider reach on paper. But at half the input price, without the refusal fallback in the middle of a long run, Opus 5 is the one I keep pointing at my own code.

Ask me again in three weeks when my bug set has more data. That is the only number here I will fully vouch for, because it is the only one I ran myself.

FAQ

Sources

Anthropic — Claude Opus 5 announcement
Anthropic — Claude Fable 5 & Mythos 5 system card
Anthropic — models overview (pricing, context windows, model IDs)
Frontier-Bench v0.1 announcement
OpenAI — Why SWE-bench Verified no longer measures frontier coding capabilities
OpenAI — Separating signal from noise in coding evaluations
Goodhart's Law