Complexity Dependent Error Rates for Physics-informed Statistical Learning via the Small-ball Method

Abstract:Physics-informed statistical learning (PISL) integrates empirical data with physical knowledge to enhance the statistical performance of estimators. While PISL methods are widely used in practice, a comprehensive theoretical understanding of how informed regularization affects statistical properties is still missing. Specifically, two fundamental questions have yet to be fully addressed: (A) what is the trade-off between soft penalties and hard constraints, and (B) what is the statistical gain of incorporating physical knowledge compared to purely data-driven empirical error minimisation. In this paper, we address these questions for PISL in classes of functions whose convex hull satisfies a small-ball condition, in particular convex classes, under physical knowledge expressed as linear equations by developing appropriate complexity dependent error rates based on the small-ball method. We specialise them to linear classes of finite dimension $q$ in which the space of solutions of the equation has dimension $k < q$, and show that (A) the physics-informed estimator attains the error bound of the hard-constrained empirical error minimiser whenever the penalty parameter is above an explicit threshold, and that (B) the penalty improves the error rate from $\sqrt{q/n}$ to $\sqrt{k/n}$ under a second moment condition on the noise only, having the effect of a dimensionality reduction. We also quantify how these conclusions degrade when the equation is misspecified, in terms of the excess residual of the target function. This work establishes a theoretical framework for evaluating the statistical properties of physics-informed estimators in such classes, contributing to closing the gap between statistical theory and practical PISL.

Machine Learning (stat.ML); Machine Learning (cs.LG); Statistics Theory (math.ST)

arXiv:2510.23149 [stat.ML]

(or arXiv:2510.23149v2 [stat.ML] for this version)

https://doi.org/10.48550/arXiv.2510.23149 Focus to learn more arXiv-issued DOI via DataCite

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.