Summary: Can AI agents conduct open-ended AI research? Early evidence from two case studies

Summary (Overview)

• Core finding: Frontier AI agents (Claude Opus 4.8 with extra-high reasoning on OpenClaw) could complete all engineering tasks required for AI research—including literature reviews, GPU debugging, experiment execution, and LaTeX compilation—but failed to produce research at the caliber of top ML conference submissions. Both papers were unambiguously rejected by the original authors (scored 2/6 and 1/6).

• Methodological contribution: The paper introduces "shadow evaluations"—a third evaluation paradigm for measuring AI R&D automation, where agents tackle the central research question of an unpublished paper and the original authors grade the output as conference reviewers. This combines open-ended tasks with expert grading while avoiding contamination from training data.

• Five recurring failure modes identified: (1) poor judgment about the bar for publishable research, (2) uncreative responses to shortcomings in research design, (3) ineffective backtracking from dead ends, (4) poor resource awareness (agents used <50% of API budgets), and (5) instruction drift/context rot.

• Robustness: A follow-up experiment using GPT-5.6 Sol Ultra with Codex scaffold reproduced nearly all failure modes, suggesting results are not merely artifacts of a specific scaffold deficiency.

• No reward hacking detected: Despite concerns, agents showed the opposite tendency—starting with more marketable claims and retiring them in favor of negative results when evidence didn't support them.

Introduction and Theoretical Foundation

The paper addresses one of the most consequential open questions about AI capabilities: whether AI agents can conduct AI research. This question underpins many forecasts of explosive AI progress (Kokotajlo et al., 2025; Cunningham et al., 2026) and is explicitly claimed by leading AI labs—Anthropic's "When AI Builds Itself" post and OpenAI's claim that GPT-5.6 Sol helped post-train a smaller model.

The evidence gap: Most existing evaluations fall into two categories with fundamental limitations:

1. Verifiable task benchmarks (RE-Bench, MLE-Bench, PostTrainBench, MLS-Bench): Agents improve a fixed, narrow metric with automatic verification. These are objective and scalable but exclude open-ended research, where agents must choose hypotheses, decide what evidence would settle a question, and recognize when to abandon approaches.

2. Blind peer review of AI-generated papers (AI Scientist-v2, Zochi): These test open-ended research but suffer from peer review being "overstretched, stochastic, and suffers from poor review quality." NeurIPS experiments (2014, 2021) showed half of review score variation was subjective, with ~25% disagreement on accept/reject decisions.

Shadow evaluations address these gaps by combining:

• Open-ended tasks (real research questions)
• Uncontaminated questions (papers not yet public, so agents can't memorize answers)
• Expert grading (original authors with deep expertise on the exact question)

The design also mirrors the real-world mechanism of recursive self-improvement: researchers delegating entire projects to agents and judging whether returned results advance their work.

Methodology

Experimental setup:

• Two unpublished NeurIPS 2026 papers used as test cases:
  • Personas paper: Structure and controllability of LLM personas (UK AI Security Institute)
  • TabPFN paper: Distribution shift detector for tabular foundation models (University of Toronto)
• Agent configuration: Claude Opus 4.8 with extra-high reasoning on OpenClaw scaffold
• Resources: Six days wall-clock time, $3,000 in Anthropic API credits, GPU credits, full VM access, and open web access
• Goal: Produce a paper worthy of publication at a top-tier AI conference

Agent capabilities provided:

• Full Linux VM on AWS with real-time monitoring of API spend, compute budget, and remaining time
• Subagent delegation with running research log
• AI self-review subagent (only sees finished PDF and NeurIPS review template)
• Three external AI reviewing tools: Stanford Agentic Reviewer, CMU Paper Reviewer, refine.ink

Human interventions (all documented):

1. Scaffold bug fix for Anthropic reasoning models
2. 24-hour deadline extension (agents submitted drafts with self-review of "Weak Reject")
3. Request for readability rewrite (papers were "inscrutable")

Survey design: Twelve coauthors provided predictions before the experiment, with low confidence and substantial disagreement, enabling assessment of how well results matched priors.

Robustness check: The TabPFN experiment was rerun with GPT-5.6 Sol Ultra using Codex scaffold, with the same time and API budgets.

Empirical Validation / Results

Expert reviews of agent outputs:

Key findings:

1. Engineering capability confirmed: Agents completed large literature reviews, debugged GPU environments, ran hundreds of experiments, retrieved external reviews via web/email, and compiled full camera-ready LaTeX documents—all without manual intervention.

2. Lack of judgment: Both agents falsified their initial hypotheses using "small, hand-curated or synthetic datasets" and presented "underpowered negative results as substantive findings." Viet Nguyen noted: "Upon testing a few unsuccessful signals... going from there to 'there are no signals we can use' is a huge leap, a kind of 'proof by example' fallacy."

3. Premature commitment: The Personas agent budgeted 42 hours for exploration but coalesced around a method after only 5 hours. The TabPFN agent committed to its headline finding 40 hours early.

4. Generator-verifier gap: AI self-reviews never returned an acceptance across ~15 rounds of revision, yet agents didn't treat this as a signal to rethink their approach—they narrowed claims and added caveats instead.

5. Resource mismanagement: Both runs ended with <50% of API budget spent. The agent could monitor usage in real-time and was encouraged to use remaining budgets.

6. Instruction drift: Both final papers exceeded page limits (content on 10th page vs. 9-page limit); the Personas paper had no visualizations vs. 15 in the original; references were 16 vs. 52 (Personas) and 36 vs. 69 (TabPFN).

Robustness experiment (GPT-5.6 Sol Ultra with Codex):

• Reproduced nearly all failure modes
• Exhausted $3,000 token budget in just over 2 days, leaving ~100 hours unused
• Did improve on one aspect: found and used a real-world distribution-shifted dataset (vs. synthetic-only in main runs)
• Self-review process continued to return rejects appropriately

Safety findings:

• No reward hacking detected; instead agents retired marketable claims in favor of honest negative results
• One access token was committed to a repository (security concern)
• Five instances of subagent hallucination were caught by the orchestrator agent

Theoretical and Practical Implications

For AI R&D automation forecasts:

• The results provide early evidence that "today's frontier models cannot solve weeks-long, open-ended AI research questions"
• However, the paper carefully notes that "the path to AI R&D does not require full automation of open-ended tasks"—narrow improvements on verifiable tasks might still accelerate research
• The distinction between "broad" and "narrow" AI capabilities (Elasticity Institute) is central to interpreting these results

For evaluation methodology:

• Shadow evaluations offer a complementary third paradigm alongside verifiable benchmarks and blind review
• The method tests the exact mechanism of recursive self-improvement: delegation and judgment of returned results
• Limitations include small sample size, non-blind reviewing, and researcher degrees of freedom

For agent development:

• The generator-verifier gap suggests that AI reviews could be used for reinforcement learning, but accuracy cannot be established from this study alone
• The failure to effectively use feedback (both AI and expert) points to a fundamental limitation in current agent architectures
• Engineering tasks and verifiable research tasks show much better agent performance (e.g., iOS app development success in prior work), suggesting the bottleneck is specifically in open-ended research judgment

For research practice:

• Both papers would have been desk-rejected at NeurIPS due to formatting violations
• The contrast between engineering capability and research judgment suggests a "capability cliff" that may be more fundamental than incremental improvements in model size or scaffold design

Conclusion

This paper provides early, carefully-documented evidence that while frontier AI agents can handle the engineering of AI research—debugging, running experiments, writing LaTeX—they struggle with the critical judgment and creativity components of the research lifecycle. The five failure modes (poor judgment about research quality bars, uncreative responses to feedback, ineffective backtracking, poor resource awareness, and instruction drift) were consistent across two papers, two models (Opus 4.8 and GPT-5.6 Sol Ultra), and two scaffolds (OpenClaw and Codex).

Future directions:

• Follow-up experiments on a larger set of research papers using GPT-5.6 Sol, Opus 5, and Fable 5
• Further optimization of scaffolds to understand sensitivity to scaffold and model improvements
• Testing stronger models as they become available (Anthropic deliberately limited Fable 5's R&D abilities during this study)
• Adversarial collaborations to check interpretations against raw materials (all artifacts are released)

The authors conclude that "today's agents can do the engineering of AI research, but struggle with critical parts of the research lifecycle," positioning shadow evaluations as a repeatable method for tracking progress as models and scaffolds improve.