analiza aiqbench
This research introduces a predictive Central Limit Theorem that enables the decomposition of total uncertainty into aleatoric and epistemic components for Bayes-filtered transformers. By applying this framework to TabPFN, the study demonstrates how context length influences these distinct uncertainty types while maintaining near-nominal frequentist coverage in tabular prediction tasks.
Uncertainty Decomposition for Bayes-Filtered Transformers via Bayesian Predictive Inference
arXiv:2602.04596v2 Announce Type: replace-cross Abstract: Bayes-filtered transformers are transformers meta-learned on sequences from a prior predictive distribution to approximate the corresponding posterior predictive distribution. They output total predictive uncertainty in a single forward pass but never explicitly represent a posterior distribution, making the standard route to separating aleatoric from epistemic uncertainty unavailable. We address this challenge through the lens of Bayesian predictive inference (BPI). Our main result is a predictive Central Limit Theorem (CLT) for supervised settings under conditions that are among the weakest known in the BPI literature. The CLT characterises the posterior of the limiting predictive distribution given an observed context as asymptotically Gaussian; the variance of this Gaussian quantifies epistemic uncertainty. We apply the framework to TabPFN, a Bayes-filtered transformer that is a state-of-the-art foundation model for tabular prediction. The resulting credible bands achieve near-nominal frequentist coverage as context length grows, and the decomposition largely matches standard desiderata: epistemic uncertainty shrinks with context length and is highest in sparsely observed regions within the span of the context data, while aleatoric uncertainty dominates near decision boundaries where classes overlap.
Abstract:Bayes-filtered transformers are transformers meta-learned on sequences from a prior predictive distribution to approximate the corresponding posterior predictive distribution. They output total predictive uncertainty in a single forward pass but never explicitly represent a posterior distribution, making the standard route to separating aleatoric from epistemic uncertainty unavailable. We address this challenge through the lens of Bayesian predictive inference (BPI). Our main result is a predictive Central Limit Theorem (CLT) for supervised settings under conditions that are among the weakest known in the BPI literature. The CLT characterises the posterior of the limiting predictive distribution given an observed context as asymptotically Gaussian; the variance of this Gaussian quantifies epistemic uncertainty. We apply the framework to TabPFN, a Bayes-filtered transformer that is a state-of-the-art foundation model for tabular prediction. The resulting credible bands achieve near-nominal frequentist coverage as context length grows, and the decomposition largely matches standard desiderata: epistemic uncertainty shrinks with context length and is highest in sparsely observed regions within the span of the context data, while aleatoric uncertainty dominates near decision boundaries where classes overlap.
Major revision, retitled. The quasi-martingale conditions of v1 are relaxed to weaker signed conditions; adds empirical diagnostics showing a meta-learned Beta-Bernoulli BFT satisfies the signed condition
Machine Learning (stat.ML); Machine Learning (cs.LG); Methodology (stat.ME)
arXiv:2602.04596 [stat.ML]
(or arXiv:2602.04596v2 [stat.ML] for this version)
Text integral sub licența deschisă a publicației · 218 cuvinte
Subiecte conexe
Generative Bayesian Filtering for State Estimation
arXiv · 24 iul. 2026
Bayesian 3D Steerable CNNs: Enabling Equivariance and Uncertainty Quantification Simultaneously
arXiv · 16 iun. 2026
What Do Foundation Models for Market Microstructure Learn? A Controlled Study with Synthetic Limit Order Books
arXiv · 15 oct. 2026
CORE-STACK+: Meta-Learning for Deep Stacked Generalization
arXiv · 24 sept. 2026