METR, the independent AI evaluation group, published a report on November 5 concluding that Anthropic is likely improving its models' capabilities on well-specified tasks at about twice the rate it managed between March 2024 and October 2025 — and that roughly 80% of that acceleration is due to AI itself. METR spent August 12 to October 29 working with Anthropic's internal data to reach that judgment.
Why it matters: "AI speeding up AI research" is the scenario frontier-lab safety policies are written to catch, because a compounding loop could produce sudden jumps in risky capabilities. This is the first public outside estimate built on a lab's internal numbers, and METR says there is a significant risk the pace gets faster still over the coming year. But the report also shows the limits of the current audit model: Anthropic held redaction rights, and METR says it used them on "a majority of the evidence we found most informative."
The numbers METR published
| Measure | METR's estimate |
|---|---|
| Growth in Anthropic's internal capability index (AECI) since Oct 2025 vs. before | 1.9x (95% CI 1.5–2.8x) |
| Range across robustness checks | 1.1x – 2.9x |
| Overall judgment | ~2x; METR "would not be surprised" by 1.3x or 3x |
| Share of current acceleration attributed to AI | ~80% |
| Self-reported uplift from the best internal model (staff polls) | ~12x average — METR calls this very loose evidence |
AECI is Anthropic's version of the Epoch Capabilities Index, a composite of benchmark scores across internal and released models. FourWeekMBA's write-up notes the fitted trend went from about +15 points a year before October 2025 to about +29 after, and that METR's figure is above the 1.7x in Anthropic's own Haiku 5.5 system card, mainly because METR included newer unreleased models.
METR says it is confident the speedup was not driven by faster growth in training compute — one reason it attributes most of the speedup to AI. It found no convincing bottleneck (algorithms, training or experiment compute, data) that would stop the trend over the next 12 months, though it admits its view on that changed several times.
How it was done — and what it couldn't see
About 2.5 full-time-equivalent staff worked for roughly eleven weeks, using benchmark histories, compute and algorithmic-efficiency data, headcount, lines of code, inference spend, uplift surveys and about 15 staff interviews. Interviews were mixed: by METR's count two interviewees supported more than 2x acceleration, three leaned yes, four were borderline, and the rest leaned no.
METR lists the weaknesses itself:
- No direct system access. It worked from datasets and documents, not Anthropic's systems, and did not sample non-public models.
- Benchmark saturation. One model scores above 50% on every benchmark used, and AECI may not track real-world impact.
- Errors in both directions. Non-AI factors and an unusually low historical baseline could inflate the estimate; continued acceleration or understated real-world gains could deflate it.
- Scope change. Anthropic dropped a planned check of how compute is split between capabilities and safety work partway through.
- Independence. METR was unpaid but worked on Anthropic devices with free tokens, and disclosed a team member's conflict of interest (their work was checked by a colleague).
The redaction fight
Anthropic says the redactions — covering detailed attribution estimates, bottleneck analysis, forecasts for future models and some interview material — were aimed at keeping out information that could speed up other AI developers. METR does not rule on whether that was justified, but says plainly that the public version does not achieve transparency about the evidence, so readers have to rely on METR's judgment. In its own checklist METR marks "grant disclosure rights upfront" and "direct system access" as not met.
Anthropic co-founder Sam McCandlish said on X that many of METR's conclusions match his own and called the redactions a tradeoff he thinks was right, according to FourWeekMBA. Anthropic has said it will publish its own report on where it stands against its Responsible Scaling Policy in the coming weeks.
What to watch
- Anthropic's RSP report. Whether it accepts METR's ~2x figure, and what it says it will do if acceleration crosses its own thresholds.
- Other labs. No other frontier developer has published a comparable outside assessment on internal data; this one sets a template — redaction rules included.
- Repeat measurements. A single ~11-week snapshot can't show whether 2x is a plateau or a waypoint. METR's warning is about the trend, not the level.
