aiqbench analysis
The paper tests whether an AI agent can choose its own research question, not just execute a human-chosen one, and it reports all five pre-registered runs, including the pilot, rather than only the best outputs. The evaluation is narrow: external AI reviewers judged the papers "at or near workshop level," and the claims rest on five runs from one frozen system, so readers should treat this as a transparency-focused case study rather than evidence of general research autonomy. The released system, papers, and records make the work independently checkable.
Letting AI Formulate the Research Question: Autonomous Research Agents
arXiv:2610.39620v1 Announce Type: new Abstract: Systems such as The AI Scientist have shown that AI agents can carry out and write up research, but in those demonstrations humans chose the research area and selected the outputs to show. We study letting the AI formulate the research question itself, and we report every run. Our system gives an off-the-shelf coding agent an instruction document and two skills: one for formulating the question, in which the agent diverges from recent technologies into problems of other fields, screens its candidates for known failure modes and obtains a critique from a different model; and one for peer review, in which a reviewer from another model family reruns the study from its records before the agent revises the paper. Over 13 development runs, changes to the one or two sentences that set the research area coincided with large shifts in what the candidate questions and the chosen papers were about, which led us to the area of a new technology applied to a societal problem. We froze the system after a pilot run and report five consecutive runs: the pilot, counted by a disclosed amendment to the pre-registration, and four pre-registered runs. All five runs went from question formulation to a revised, re-computable paper, with local computation on a laptop and no research input from humans, and external AI reviewers reading only the papers judged them at or near workshop level. We release the system, the five papers and the records needed to reproduce them.
Abstract:Systems such as The AI Scientist have shown that AI agents can carry out and write up research, but in those demonstrations humans chose the research area and selected the outputs to show. We study letting the AI formulate the research question itself, and we report every run. Our system gives an off-the-shelf coding agent an instruction document and two skills: one for formulating the question, in which the agent diverges from recent technologies into problems of other fields, screens its candidates for known failure modes and obtains a critique from a different model; and one for peer review, in which a reviewer from another model family reruns the study from its records before the agent revises the paper. Over 13 development runs, changes to the one or two sentences that set the research area coincided with large shifts in what the candidate questions and the chosen papers were about, which led us to the area of a new technology applied to a societal problem. We froze the system after a pilot run and report five consecutive runs: the pilot, counted by a disclosed amendment to the pre-registration, and four pre-registered runs. All five runs went from question formulation to a revised, re-computable paper, with local computation on a laptop and no research input from humans, and external AI reviewers reading only the papers judged them at or near workshop level. We release the system, the five papers and the records needed to reproduce them.
Artificial Intelligence (cs.AI); Computers and Society (cs.CY)
arXiv:2610.39620 [cs.AI]
(or arXiv:2610.39620v1 [cs.AI] for this version)
https://doi.org/10.48550/arXiv.2610.39620 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 343 words
Related coverage
At the Limits of Computability: Steps Towards Autonomous AI Discovery on the Busy Beaver Frontier
arXiv · Oct 20, 2026
DoAtlas-2: A Foundation for Self-Evolving Causal Biomedical Discovery
arXiv · Sep 29, 2026
Video2World: Benchmarking Coding Agents for Interactive World Modeling from Embodied Videos
arXiv · Oct 7, 2026
Zero-Shot Semantic Re-Identification for Autonomous Driving: A VLM Baseline Study
arXiv · Jun 9, 2026