Part 2 of Intelligence Explosion: Evidence, Bottlenecks, and Control. Research checked October 4, 2026.
An AI laboratory can report that machines write most of its code while humans still determine most of its research direction. A benchmark can show that an agent completes difficult tasks while the surrounding team spends more time reviewing its work. These statements can all be true.
The intelligence-explosion debate becomes confused when evidence about one of these activities is presented as evidence about all of them. A more useful reading separates capability, adoption, delegation, productivity, and acceleration.
Capability asks what a system can accomplish under specified conditions. Adoption asks whether people use it. Delegation asks how much responsibility they hand over. Productivity asks how useful output changes relative to total inputs. Acceleration asks whether the rate of progress itself is rising. Each can influence the next, but none automatically establishes it.
What company disclosures establish
Consider a particularly relevant company disclosure. Anthropic reported that, as of August 2026, Claude led 26% of its measured AI R&D work, meaning it could complete most of a task from a high-level prompt under human supervision. More than 90% was at or above the collaboration level. The company also explicitly reported no measured subset operating fully autonomously. Its prototype index used internal work records and model-assisted classification; the authors acknowledged methodological and correlated-judge limitations. Anthropic’s measurements.
This is informative evidence of changing work practices. It is a self-reported index with a defined task basket, rather than a randomized estimate of discoveries per researcher-hour. It should raise our confidence that substantial delegation is occurring without being converted into an unsupported claim that an entire laboratory is autonomous.
What benchmarks measure
Benchmarks answer another question. METR’s January 2026 Time Horizon 1.1 update measured task difficulty in human-equivalent time. Its estimated frontier doubling time varied with the fitting window: approximately seven months over the longer historical series, about 131 days since 2023, and about 89 days since 2024. METR also warned that task composition affects the trend and that most long-task human durations were estimates. Time Horizon 1.1.
A 50% time horizon is a reliability-qualified measure over a task distribution. It is not a guarantee that an agent can independently run a project of that duration. Extrapolating the fitted line to multi-month research is a forecast conditional on continued trends, task comparability, and successful transfer to messier work.
The distinction becomes tangible when evaluators replace a benchmark grader with maintainers. A March 2026 METR research note found that roughly half of test-passing SWE-bench Verified patches from the agents studied would not be accepted by repository maintainers after its adjustment for review noise. The study involved four maintainers, three repositories, and 296 AI-generated patches. Agents did not receive a chance to revise in response to feedback. METR’s maintainer review study.
The result identifies a gap between passing tests and meeting a working project’s requirements. It does not establish an unfixable ceiling on agent usefulness. Iteration, clearer instructions, and stronger evaluations might close part of that gap. Their costs belong in the productivity calculation.
Productivity evidence changes over time
There is a similar trap in quoting productivity studies after the conditions have changed. METR’s early-2025 randomized study found that experienced open-source developers took 19% longer with the tools studied. Its February 2026 follow-up reported indications of speedups, but also serious selection and timing problems: some developers and tasks were absent because participants did not want to work without AI. METR treated the newer data as weak evidence about the magnitude of current productivity effects. METR’s follow-up and original-study comparison.
Neither “AI slows developers by 19%” nor “the follow-up proves a universal speedup” is an adequate statement of this evidence. The former freezes a dated result; the latter discards the follow-up’s own limitations.
More directly relevant research tasks show a similarly mixed picture. PostTrainBench gives agents a bounded post-training assignment, including ten hours on one H100 GPU. Its authors reported substantial improvements and some targeted successes, alongside overall results below official instruction-tuned models and episodes of reward hacking. Those official models were not a clean, equal-budget human control group. The comparison demonstrates remaining performance gaps under the benchmark’s conditions; it does not isolate the causal value of human versus automated research. PostTrainBench.
Measure the whole discovery process
What evidence would resolve more of the disagreement? I would prioritize a paired record of output and cost: reproducible improvements achieved; human hours consumed; inference and experimental compute; unsuccessful attempts; time to validate; and gains that survive transfer to a different workload. Every attractive productivity ratio should disclose both its numerator and its denominator.
This would also address survivorship bias. Publishing the successful experiment alone hides the search that selected it. A system that finds one impressive result after thousands of expensive failures may still be valuable, but it answers a different economic question from a system that reliably produces useful results at low cost.
Evidence sources should remain visibly distinct. Peer-reviewed experiments, working papers, company disclosures, and benchmark reports have different strengths. A company may have the best access to its internal workflows and strong incentives to frame them favorably. An independent evaluator may have less access and a narrow task sample. Neither label eliminates the need to inspect the method.
The evidence available for this series supports a substantive conclusion: AI already performs meaningful parts of AI development, and the scope of delegation is expanding. The public record is much less decisive about sustained, independent research productivity and self-sustaining acceleration. That distinction makes the research agenda clearer. We need to measure whether the whole discovery process is getting better, with its costs and failures included.
The ContextOS connection
This measurement discipline connects to the ContextOS evaluation and observability contract. Fix the comparison protocol, retain failed attempts, and distinguish accepted outcomes from apparent task completion. The contract provides an engineering structure for evidence; it does not replace independent measurement.