aiqbench analysis
This research demonstrates that transformer-based models can autonomously recover fundamental physical structures, such as electrostatic interactions and cutoffs, without relying on manually engineered graph priors. The study establishes that general-purpose architectures can serve as valid baselines for scientific modeling, with explicit physical constraints introduced only when empirical necessity dictates. This clarifies the boundary between learned physical patterns and traditional inductive biases in atomistic simulations.
Transformers Discover Molecular Structure Without Graph Priors
arXiv:2510.02259v2 Announce Type: replace Abstract: Computational simulations play a central role in scientific discovery, and machine learning (ML) has emerged as a promising alternative to traditional physics-based modeling. However, scientific modeling requires physically meaningful predictions, raising a fundamental question for data-driven methods: to what extent can physical inductive biases - that is, prior assumptions about the structure of the physical world - emerge by learning from data alone? In atomistic modeling, for example, ML architectures have historically embedded strong physical inductive biases - such as geometric locality and graph structure - based on the assumption that these priors are necessary for physical predictions. We systematically develop an understanding of how physical patterns can alternatively be discovered directly from data by training a model without domain-specific priors, including any manually defined atomistic pairwise interactions. We find that the model autonomously recovers key physical structure, such as learned interatomic interaction strengths that mirror classical electrostatics and interaction cutoffs consistent with traditional physical models. We further demonstrate predictable neural scaling law behavior with increased data and compute, and find accuracy on certain metrics competitive with physics-informed architectures. Our results clarify the boundary between engineered inductive bias and learnable physical structure, suggesting that general-purpose architectures can serve as principled baselines for scientific modeling, with explicit priors introduced only when empirically necessary.
Abstract:Computational simulations play a central role in scientific discovery, and machine learning (ML) has emerged as a promising alternative to traditional physics-based modeling. However, scientific modeling requires physically meaningful predictions, raising a fundamental question for data-driven methods: to what extent can physical inductive biases - that is, prior assumptions about the structure of the physical world - emerge by learning from data alone? In atomistic modeling, for example, ML architectures have historically embedded strong physical inductive biases - such as geometric locality and graph structure - based on the assumption that these priors are necessary for physical predictions. We systematically develop an understanding of how physical patterns can alternatively be discovered directly from data by training a model without domain-specific priors, including any manually defined atomistic pairwise interactions. We find that the model autonomously recovers key physical structure, such as learned interatomic interaction strengths that mirror classical electrostatics and interaction cutoffs consistent with traditional physical models. We further demonstrate predictable neural scaling law behavior with increased data and compute, and find accuracy on certain metrics competitive with physics-informed architectures. Our results clarify the boundary between engineered inductive bias and learnable physical structure, suggesting that general-purpose architectures can serve as principled baselines for scientific modeling, with explicit priors introduced only when empirically necessary.
Machine Learning (cs.LG); Materials Science (cond-mat.mtrl-sci); Chemical Physics (physics.chem-ph); Biomolecules (q-bio.BM)
arXiv:2510.02259 [cs.LG]
(or arXiv:2510.02259v2 [cs.LG] for this version)
https://doi.org/10.48550/arXiv.2510.02259 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 312 words
