METR says Anthropic’s capability gains are running at about twice the old pace, and most of the evidence is redacted
The assessment, dated Thursday, covers 12 August to 29 October. METR’s public estimate is about 2× the March 2024–October 2025 rate on well-specified tasks, with a range that could be 1.3× or 3×. Anthropic redacted a majority of the evidence METR found most informative. Sam McCandlish says that was to avoid handing competitors a roadmap.
Deep dive5 minIn-depth
Priya Shah5 min read
- Anthropic
- METR
- Safety
- Research

Photo: METRSource page
METR on Thursday published an assessment of whether Anthropic’s own AI is speeding up Anthropic’s research. The post is dated 5 November. The work ran from 12 August to 29 October, with a staff METR describes as 2.5 full-time equivalents. The public conclusion, which METR labels as high-uncertainty, is that as of October Anthropic was likely improving its models on well-specified tasks at about twice the rate it did between March 2024 and October 2025. METR says it would not be surprised if the true figure were closer to 1.3 times or 3 times that earlier rate. A majority of the acceleration now, it says, is likely due to Anthropic’s use of AI for research and development. Other causes, it says, likely played a large role in late 2025.
The number METR will put a confidence interval on is narrower than that judgment. Since October 2025, scores on AECI — Anthropic’s version of the Epoch Capabilities Index — have grown at an estimated 1.9 times the pre-October 2025 rate, with a 95 percent confidence interval of 1.5 to 2.8. METR reads that as roughly two years of historical benchmark progress in the past year, about as much as from October 2023 to October 2025, which it glosses as Claude 2.1 to Sonnet 4.5. The estimate is higher than the 1.7 times reported in the Haiku 5.5 system card, METR says, mainly because the new calculation includes newer models that have not been released. Robustness checks, varying which benchmarks and models are included, produce acceleration rates from 1.1 times to 2.9 times. METR says it would be surprised if an improved version of the index showed no acceleration, or acceleration above 3 times.
What Anthropic let out, and what it crossed out
Anthropic had the right to redact non-public information. METR says it could say whether that right was used, and whether findings depended on material the public cannot see. Anthropic used it. METR writes that it could share its highest-level conclusions about risk, and that Anthropic redacted a majority of the evidence METR found most informative for those conclusions. Anthropic’s description of the cuts, as METR quotes it, was that they were “aimed at removing confidential information that could accelerate other AI developers.” METR says much of the redacted evidence had more than one plausible reading, that both sides are uncertain how to put it together, and that the public therefore has to rely on METR’s judgment more than METR thinks is ideal.
We redacted a lot, and I accept that this makes it harder for others to check METR’s work. But we determined that sharing exactly what is driving the speedup isn’t the right thing to do, both because it’s commercially sensitive and because it risks handing out a roadmap for others to go faster.
— Sam McCandlish, an Anthropic co-founder, on X, 5 November 2026
McCandlish’s post, timestamped 7:20 p.m. that day, says many of METR’s conclusions match his own. “AI is speeding up our research, but it’s hard to say exactly how much.” He says Anthropic gave METR benchmark scores for unreleased models, estimates of how models are progressing, and interviews with researchers, and that METR had independence to reach its own conclusions. He also says AI cannot yet make the most important contributions to model training on its own, and that METR thinks the pace could, by default, accelerate further. In the coming weeks, he writes, Anthropic plans to publish its own analysis of the speed of progress inside the company, including where it stands against its Responsible Scaling Policy, and new safety and security steps. Jay Peters at The Verge, in a note timestamped 7:22 p.m. UTC on Thursday, reports the same plan and the same twice-the-old-rate finding. Both McCandlish and METR say this review is separate from the embedded third-party evaluators Anthropic described on 15 October.
The part METR will attribute to AI
METR says it is confident the acceleration was not caused by a faster increase in training compute. Further detail on that evidence was redacted. Beyond compute, it says Anthropic’s own staff did not agree, and that METR did not find decisive evidence. Its guess is that a majority — “perhaps ≈80%” — of the October 2026 acceleration is due to AI, with substantial uncertainty, and that AI’s share has grown more important over time. The more detailed estimates of AI’s effect on the training pipeline were redacted.
What remains in the public post is softer evidence, and METR marks it as such. Interviews with researchers, it says, centered around a 2 times acceleration, with a wide range. Slack polls, in which staff react to vote on how much the latest model speeds them up, reached 12 times in October 2026. METR calls those polls very loose evidence: low-effort answers, likely an unrepresentative subset, and a form of self-report that can overstate real productivity. It also says a reported personal speedup is not the same thing as a faster rate of progress for the lab. Researchers told METR that the work is still mostly assistance, not fully autonomous contributions to the most important parts of training, with “signs of life” that the autonomous share could grow, varying by area.
One internal system is described by name. Model 4, METR writes, finished training in September and October 2026 and scores above 50 percent on all 15 AECI benchmarks it was tested on. Its AECI score in the post is 178.6. Dropping three benchmarks whose fits looked flat or concave raises that score by a point, to 179.7. METR says the lack of benchmarks that are still hard for Model 4 makes it nervous about measuring the index from here. It dates models by when most Anthropic employees could use them, not by public release, and says those dates show slightly higher acceleration because recent models are being held longer before release.
On bottlenecks, METR says it did not see evidence that algorithmic progress, training compute, experiment compute, or data would block further gains in the next 12 months. It also says its view of those bottlenecks changed several times as more detail arrived, that it is not confident, and that Anthropic redacted the analysis of a number of specific bottlenecks. The original contract included a check of how much compute went to capabilities versus safety. Anthropic dropped that piece midway, before METR had reached substantive conclusions on compute use, and said it wanted the engagement focused on risk assessment for now and would revisit the allocation check later. METR did not get to sample non-public models. It was not given the kind of access an employee would have. Staff worked on Anthropic devices. Anthropic provided free tokens, which METR says it used mainly for data analysis. METR says it was not paid. The final public report was submitted on 3 November, after a short extension to apply tighter redaction guidance at the end of October.


