analiza aiqbench
This research provides a rigorous empirical analysis of Prior-Data Fitted Networks (PFNs), decomposing their deployment error into prior, context, and amortization components to reveal persistent uncertainties in rare settings. The study demonstrates that while PFNs generally approximate Bayesian posteriors well, independent pretraining can lead to disagreement at inference time, which is effectively mitigated through specific Bayesian model averaging techniques. These findings offer critical insights for practitioners relying on amortized inference methods to manage predictive uncertainty in complex models like TabICLv2.
How Well Do Prior-Data Fitted Networks Amortize Bayesian Inference?
arXiv:2610.13764v1 Announce Type: cross Abstract: Prior-data fitted networks (PFNs) are trained to approximate the Bayesian posterior predictive distribution (PPD) of a synthetic prior, yet results about the approximation quality are surprisingly thin. Taking an empirical risk minimization perspective, we decompose the deployment error of PFNs into prior, context, and amortization error, and measure the latter against a reference PPD under a realistic prior. Results show that PFNs generally recover the targeted PPD very well. Nevertheless, we find that parts of the residual amortization error persist, especially in \textit{rare} settings under the prior pretraining distribution, even when scaling the model size, which manifests in disagreement between independently pretrained PFNs and leaves uncertainty about the predictor at inference time. As a remedy, we study multiple types of Bayesian model averages and find that these can consistently reduce this uncertainty and improve predictions up to full-size TabICLv2 models.
Title:How Well Do Prior-Data Fitted Networks Amortize Bayesian Inference?
Abstract:Prior-data fitted networks (PFNs) are trained to approximate the Bayesian posterior predictive distribution (PPD) of a synthetic prior, yet results about the approximation quality are surprisingly thin. Taking an empirical risk minimization perspective, we decompose the deployment error of PFNs into prior, context, and amortization error, and measure the latter against a reference PPD under a realistic prior. Results show that PFNs generally recover the targeted PPD very well. Nevertheless, we find that parts of the residual amortization error persist, especially in \textit{rare} settings under the prior pretraining distribution, even when scaling the model size, which manifests in disagreement between independently pretrained PFNs and leaves uncertainty about the predictor at inference time. As a remedy, we study multiple types of Bayesian model averages and find that these can consistently reduce this uncertainty and improve predictions up to full-size TabICLv2 models.
Machine Learning (stat.ML); Machine Learning (cs.LG)
arXiv:2610.13764 [stat.ML]
(or arXiv:2610.13764v1 [stat.ML] for this version)
https://doi.org/10.48550/arXiv.2610.13764 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 246 cuvinte
Subiecte conexe
PhyloGFN: Phylogenetic inference with generative flow networks
arXiv · 1 sept. 2026
Synthesis Superintelligence: from Semiconductors to Superconductors — Periodic Labs’ Liam Fedus and Ekin Dogus Cubuk
Latent Space · 8 oct. 2026
ProximalFM: Amortized Proximal Causal Inference under Hidden Confounding
arXiv · 7 oct. 2026
Frequentist Consistency of Prior-Data Fitted Networks for Causal Inference
arXiv · 23 iul. 2026