Do Tabular Foundation Models Know Physics? Contamination, Units, and the Deterministic Limit

Title:Do Tabular Foundation Models Know Physics? Contamination, Units, and the Deterministic Limit

Abstract:Tabular foundation models (TFMs) learn to fill in tables the way language models fill in text, and tables are arguably the format in which most physical measurement arrives. Did they learn any physics in the process? They are Bayesian by construction, so the question is what their prior contains. We probe it directly, evaluating four of them (TabPFN-3, TabICLv2, TabDPT and Real-TabPFN-2.5) against six baselines on datasets sampled from 316 physical equations, in and out of domain. TFMs dominate, out of the box and after tuning. But we show that their prior can represent neither a noiseless mechanism nor physical units, which is why they interpolate physics without yet being able to act as physical models.

5 pages (4 figures, 1 table). Submitted to the Representations for the Physical Sciences Workshop @ NeurIPS 2026

Machine Learning (cs.LG); Instrumentation and Methods for Astrophysics (astro-ph.IM)

arXiv:2609.02766 [cs.LG]

(or arXiv:2609.02766v1 [cs.LG] for this version)

https://doi.org/10.48550/arXiv.2609.02766 Focus to learn more arXiv-issued DOI via DataCite (pending registration)

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.