Safe(r) Self-Driving Labs #1

Context: This is inspired by my first ever piece I wrote on anything AI safety related when I discovered AI safety was a thing back in March 2026, published originally on my Substack. The version here has been modified to represent more accurately the current state of the field, which is continuously expanding (for good and bad). This piece reflects my views based on having worked first hand in a wet lab, my experience in AI4Chemistry, and being deeply involved in the automation community for a while. The conversation is much bigger than my initial attempt, and I'm planning to revisit it periodically (hence the #1 in the title). Disclaimer: Automated labs (not autonomous) are complex, and people spend literally years trying to figure out how to adapt different manual workflows to Opentrons etc; add agents to them (to get to the autonomous) and you get an insane framework that you need to both optimise, as well as make sure it's done so safely. There are multiple layers to this, including robotics, cyber, CBRN etc. threats, and they're more often than not overlapping. Here, I am focusing on the safety and security aspects from a physical and biological sciences perspective (i.e. the application space), but I might mention some of these overalps with other threat modes at times. An important consideration when assessing potential CBRN threats is the access - given an LLM could tell us how to make a bomb or a drug, realistically how easily would it be to access the right apparatus and tools to manufacture it? At the moment, the answer is closer to 'not likely at all' - and even in the current advanced labs, going from paper (or screen) idea to practice takes advanced capabilities. But I feel like this could dramatically shift as self-driving labs (SDLs) are becoming more and more capable. Self-driving platforms, robotic workflows, and AI-guided discovery pipelines are actively being deployed in academic and industrial settings towards the unified goal and application space of ‘closing the loop’ in science - to understand the landscape better, I couldn't recommend this SDL map enough. Although automation has been around for a while, the prospect of introducing agentic workflows in their operations is very appealing as it would remove, or at least reduce, the human feedback layer, and make very complicated - and often dangerous - processes much safer. In simpler terms, an SDL is a cyber-physical system: a model's decisions can be translated into physical actions, and mistakes that would normally remain as text on a screen (if caught in due time) can instead have physical consequences. The more authority an agent has over equipment, reagents, and experimental decisions, the more important this distinction becomes. The question of how to assess the safety of these systems before and during deployment (and even after) is therefore becoming urgent - and I would argue we need frameworks bridging both software AND standard lab safety practices. Where does risk actually come from? Risk vectors can be classed in three distinct categories based on size and impact potential: In silico prediction This covers everything the predictive pipeline determines before any physical action occurs, including: the compounds it targets the precursors it identifies their toxicity and environmental persistence the synthesis and purification processes it designs the application space it opens up (i.e. a certain ligand could be made to be used further as part of drug synthesis, but on its own could be a pesticide - the context of deployment can be very important especially when tuning the scale of the process) the specific actions the hardware should undertaken, given everything goes well Critically, some of these risks are intrinsic to the task itself - they would exist regardless of which model is used. Others, such as the waste management and interaction with environment depend on the specific procedure and conditions used. A subtle but important sub-risk here is the simulation-to-reality gap (known as Sim2Real, the subject of many labs in both materials and drug discovery at the moment). The size of this gap depends heavily on the quality and representativeness of the data available to the model. A model trained on relatively consistent, high-quality literature data may perform well in simulation but fail to account for the variability encountered in a real laboratory - for example, impurities in technical-grade reagents, differences between equipment, or poorly documented experimental conditions. These discrepancies could lead to unexpected reaction behaviour (including runaway exothermic reactions or toxic by-product formation). Data provenance therefore matters enormously, but is rarely documented in sufficient detail. Lab execution Even a model that reasons correctly about chemistry can introduce risk through poor hardware awareness (and recent work suggests the latter is actually the current big bottleneck in automated la