SemiAnalysis’s new agent benchmark puts the best model, Claude Opus 5.5, at 12 percent
GDPx, published Monday, has 206 tasks across 11 domains. Gemini 4 Carbon is second at 10.9 percent and GPT-6 Astra third at 9.6. The authors say safety classifiers hold Opus and Astra down in three domains, and that binary grading on finance and legal gives every model a zero.
Feature4 minIn-depth
Priya Shah4 min read
- Benchmarks
- Claude
- Gemini
- GPT

Photo: SemiAnalysisSource page
SemiAnalysis on Monday published GDPx, a benchmark of long agent tasks, and the public leaderboard puts the best score at 12.0 percent. That is Claude Opus 5.5. Gemini 4 Carbon is second at 10.9 percent, and GPT-6 Astra is third at 9.6 percent. Max Kan, Andrew Megalaa and Dylan Patel write that it is “a true frontier-level benchmark: even the best model scores just 12%.” The site counts 206 tasks, 11 domains and 14 partner companies.
The domains are chip design, cybersecurity, biology, finance, legal, quant trading, recursive self-improvement, hardware design, materials science, alignment and software engineering. The authors say they left out government, real estate and insurance, because they wanted the fields where they expect frontier models to produce revenue, rather than a sample of the whole economy. The default time limit is eight hours, though some tasks are shorter. Models ran in their own harness where SemiAnalysis had one: Claude Code for Opus 5.5, Codex for GPT-6 Astra, Antigravity for Gemini 4 Carbon, and OpenCode for the rest. The headline figure is pass@1 averaged over three attempts. Each domain has the same weight, and so does each partner inside a domain.
A different winner in almost every field
The note says Opus 5.5 is the best all-around model and leads quant trading, hardware design and chip design. Gemini 4 Carbon is the best biology model and slightly ahead in software engineering. The authors call Carbon a real improvement from Google and the only model in the same range as Opus and Astra, and they write that a wider impression will have to wait until Carbon is generally available. GPT-6 Astra leads recursive self-improvement and materials science. Muse Wonder Full, they write, is “the best available cyber model today by a meaningful margin.”
The public comparison lists headline scores for 13 models. After the top three come Muse Wonder Full at 9.0, Claude Fable 5.1 at 8.5, GPT-6.1 Sol at 8.4, Claude Sonnet 5.5 at 8.2, Gemini 4 Argon at 7.3, Muse Wonder at 6.8, DeepSeek 4.1 Flash 1017 at 5.4, GPT-6 Luna at 5.1, Grok 4.7 at 3.9 and GLM 5.3 at 2.7. A cost chart on the same page says it is comparing 20 models, and it leaves Muse Wonder Full off the plot because the model has no public price. In the note, Kan, Megalaa and Patel say Luna and DeepSeek sit at the cheap end, Opus at the expensive end, and OpenAI’s models in the middle. They do not print a dollar cost per task.
Two caveats sit next to the headline. The authors write that Opus 5.5 and GPT-6 Astra have lower overall scores because safety classifiers interfere on cybersecurity, biology and alignment. A second ranking drops those three domains. The alternate scores are in a chart, and they are not printed as numbers in the text. Finance and legal are a separate problem. The note says the tasks are hard enough that binary grading gives every model a zero, so the leaderboard uses an average partial score instead. In one public finance task, a quality-of-earnings workbook, SemiAnalysis says that across 25 attempts no model solved it. GPT-6.1 Sol averaged 5.6 of 11 checks, Opus 5.5 and GPT-6 Astra averaged 5.4, and GLM 5.3 and Grok 4.7 averaged under 2. The best single run passed 8 of 11. In one of the errors they describe, 10 of 15 runs by Opus, Astra and Sol treated a waived $418 premium-audit adjustment as settling an old workers’ compensation policy, and understated trailing twelve-month EBITDA by $4,728.
A data room, not a prompt with the answer attached
The authors set GDPx against OpenAI’s GDPval, which they date to 25 September 2025. They say a GDPval task hands the model a fully specified prompt and the files it needs, and asks for a document in one shot. A GDPx finance task, from Living Context, is built out of a real transaction and then anonymized. In the example they publish, a fund they call Bannock Capital buys about 62 percent of a company they call Haverly Snack Company for $880 million plus an earnout. The names are fake. The environment has about 4,000 files, and the agent has to find the ones that matter. Living Context’s finance set, the note says, is 24 tasks. Extrapolate supplied the 10 tasks on recursive self-improvement.
The other partners named in the note are Aria, Arcverex and Ergodex for chip design, Arithmetic for cybersecurity, LatchBio for biology, Chakra for legal, Third Moment for quant trading, Beautiful Machines for hardware design, ValenceForge and Emulated for materials science, Fleet for alignment, and Datacurve for software engineering. SemiAnalysis says every partner had to pass an automated quality check on the first try, with minor mistakes at most, and that the same check flags nearly half the tasks in Terminal Bench 4.0 as broken. Fireworks provided inference for the open-weight models, and Modal provided the sandboxes. The per-domain scores beyond the winners named in the text are charts on the site.


