analiza aiqbench
Anthropic's recent enzyme discovery highlights the growing intersection of AI agents and life sciences research, yet the lack of technical transparency regarding model architecture and compute resources limits the scientific community's ability to fully evaluate the breakthrough. The episode underscores a critical tension between publicizing AI achievements and maintaining rigorous validation standards necessary for complex biological systems where simple mathematical prompting is insufficient.
Anthropic shares an exciting result in enzyme discovery - and an exercise in public's perception of science and AI
Disclaimer: I'm not affiliated with Anthropic, these are my own thoughts as a former wet lab chemist currently working in chem-biosecurity and AI evals. I appreciate the complexity of scientific research and genuinely believe AI could play an important role in how we do science in the decades to come - but I have some concerns on how these labs, the media, and even the community, perceive and share this kind of news. I am basing my opinion on the announcement on X, and their official release on their website. Since posting this, a preprint came out, this is not considered below in my piece - and I think in a way that makes it even more relevant. TL;DR 'Agent X did this' and 'humans used agent X to do this' are two VERY different stances. Please stop using them interchangeably! Yesterday, 23rd Sep '26, Anthropic shared that their in-house Life Science unit made a new discovery in biology, aided by Claude - a previously unknown enzyme system hidden in the DNA of bacteriophages. The headline numbers are surprisingly small for the scale of the search - roughly 950 agents, 210 million tokens and 21 hours - although without more detail on the models, harness, search space and compute, those numbers are difficult to interpret. And it's not the only thing missing (especially for skeptics like me). Amodei acknowledged in his tweet himself that biology isn't maths, and you can't just prompt the AI to solve an equation in biology and you can cure diseases - life sciences are, by definition, revolved around life. A prediction without a robust validation protocol is just a number on your screen. I don't think I can emphasise this enough!!!!! They have clearly done some experimental validation, but the public announcement gives surprisingly little detail about what was actually demonstrated experimentally, how robust those results are, and which parts of the claimed biological function remain hypothetical (i.e. enzymes behave differently when analysed on their own or as part of the system they are part of, and that requires different approaches). Sharing this breakthrough to the public without explaining how this result is valid, especially as Anthropic being knowingly one of the biggest frontier labs in AI, makes careful framing especially important when communicating the result publicly. To be clear, I am not saying this result isn't valid. To give them the benefit of the doubt, it might be the case that they want to a. patent it, b. do more tests, c. want to understand more adjacent aspects of this new space, d. first send it to a big journal and are awaiting peer review (and some will take a while to get back to you), or e. simply choose not to release all of the details to the public. All of these are, in my opinion, perfectly reasonable causes for the lack of clarity we received with the news - and this is the reality of how science is conducted both in the industry and academic labs. However, what I don't agree with is the decision to publicly announce the result while providing relatively little detail on the experimental validation, and overemphasise (to my perception) Claude's/ AI role. In one of the posts, it is stated that biologists used Claude, 'which works through data and literature to generate hypotheses and candidate biological systems to study'; in a later one, they state that 'Claude discovered...'. These are two EXTREMELY different scenarios, and using them interchangeably can do a lot of harm to the scientific and AI4Science communities. From what they've shared so far, this looks somewhere between the two: Claude appears to have played a substantial role in hypothesis generation and candidate selection, while humans were still responsible for experimental validation and interpretation. And that's great. This is already a massive timeline shrink for traditional scientific discoveries, and a solid use case for researchers to integrate more AI-based tools in their work, to make it safer, faster and more efficient. This distinction really matters. Finding a previously unknown system is itself a meaningful scientific result, but identifying a candidate system, demonstrating that it exists, establishing what it does, and understanding why it does it are different scientific achievements. Calling the whole process “Claude discovered...” collapses those distinctions. But the type of language and discourse they chose in this press release could lead some people who don't come from a science/ research background per se to get the wrong impression that we've reached a point where AI can surpass some of the most capable human experts in traditionally specialisation and time intensive fields - which I don't think is the case at the moment. Discuss
Disclaimer: I'm not affiliated with Anthropic, these are my own thoughts as a former wet lab chemist currently working in chem-biosecurity and AI evals. I appreciate the complexity of scientific research and genuinely believe AI could play an important role in how we do science in the decades to come - but I have some concerns on how these labs, the media, and even the community, perceive and share this kind of news.I am basing my opinion on the announcement on X, and their official release on their website.
Disclaimer: I'm not affiliated with Anthropic, these are my own thoughts as a former wet lab chemist currently working in chem-biosecurity and AI evals. I appreciate the complexity of scientific research and genuinely believe AI could play an important role in how we do science in the decades to come - but I have some concerns on how these labs, the media, and even the community, perceive and share this kind of news.
I am basing my opinion on the announcement on X, and their official release on their website.
Yesterday, 23rd Sep '26, Anthropic shared that their in-house Life Science unit made a new discovery in biology - a previously unknown enzyme system hidden in the DNA of bacteriophages. The headline numbers are surprisingly small for the scale of the search - roughly 950 agents, 210 million tokens and 21 hours - although without more detail on the models, harness, search space and compute, those numbers are difficult to interpret. And it's not the only thing missing (especially for skeptics like me).
Amodei acknowledged in his tweet himself that biology isn't maths, and you can't just prompt the AI to solve an equation in biology and you can cure diseases - life sciences are, by definition, revolved around life. A prediction without a robust validation protocol is just a number on your screen. I don't think I can emphasise this enough!!!!! They have clearly done some experimental validation, but the public announcement gives surprisingly little detail about what was actually demonstrated experimentally, how robust those results are, and which parts of the claimed biological function remain hypothetical (i.e. enzymes behave differently when analysed on their own or as part of the system they are part of, and that requires different approaches). Sharing this breakthrough to the public without explaining how this result is valid, especially as Anthropic being knowingly one of the biggest frontier labs in AI, makes careful framing especially important when communicating the result publicly. To be clear, I am not saying this result isn't valid. To give them the benefit of the doubt, it might be the case that they want to a. patent it, b. do more tests, c. want to understand more adjacent aspects of this new space, d. first send it to a big journal and are awaiting peer review (and some will take a while to get back to you), or e. simply choose not to release all of the details to the public.
All of these are, in my opinion, perfectly reasonable causes for the lack of clarity we received with the news - and this is the reality of how science is conducted both in the industry and academic labs. However, what I don't agree with is the decision to publicly announce the result while providing relatively little detail on the experimental validation, and overemphasise (to my perception) Claude's/ AI role. In one of the posts, it is stated that biologists used Claude, 'which works through data and literature to generate hypotheses and candidate biological systems to study'; in a later one, they state that 'Claude discovered...'. These are two EXTREMELY different scenarios, and using them interchangeably can do a lot of harm to the scientific and AI4Science communities.
From what they've shared so far, this looks somewhere between the two: Claude appears to have played a substantial role in hypothesis generation and candidate selection, while humans were still responsible for experimental validation and interpretation. And that's great. This is already a massive timeline shrink for traditional scientific discoveries, and a solid use case for researchers to integrate more AI-based tools in their work, to make it safer, faster and more efficient. This distinction really matters. Finding a previously unknown system is itself a meaningful scientific result, but identifying a candidate system, demonstrating that it exists, establishing what it does, and understanding why it does it are different scientific achievements. Calling the whole process “Claude discovered...” collapses those distinctions. But the type of language and discourse they chose in this press release could lead some people who don't come from a science/ research background per se to get the wrong impression that we've reached a point where AI can surpass some of the most capable human experts in traditionally specialisation and time intensive fields - which I don't think is the case at the moment.
Text integral sub licența deschisă a publicației · 814 cuvinte
