aiqbench analysis
GOTabPFN addresses the High-Dimensional, Low-Sample Size (HDLSS) challenge by introducing a graph-guided feature ordering method that avoids retraining large backbones. By compressing locally adjacent features into meta-features, the approach significantly improves stability and accuracy for TabPFN-style models under tight token budgets. This work offers a practical pathway for deploying compact foundation models in data-scarce tabular regimes without the computational cost of massive pre-trained weights.
GOTabPFN: From Feature Ordering to Compact Tokenization for Tabular Foundation Models on High-Dimensional Data
arXiv:2606.05441v1 Announce Type: cross Abstract: We investigate how to make small tabular foundation models effective for High-Dimensional, Low-Sample Size (HDLSS) tabular prediction without retraining large backbones. We introduce Graph-guided Ordering with Local Refinement (GO-LR), show its equivalence to weighted Minimum Linear Arrangement, and interpret the practical solver as a TSP-path-style surrogate. We propose GOTabPFN,which builds on GO-LR, and a Neuro-Inspired Subunit Compression (NSC) unit to pool locally adjacent ordered features into meta-features, yielding a compact representation that makes TabPFN-style prediction practical in HDLSS regimes. Across tabular benchmarks, GOTabPFN improves stability and accuracy under tight token budgets.
Abstract:We investigate how to make small tabular foundation models effective for High-Dimensional, Low-Sample Size (HDLSS) tabular prediction without retraining large backbones. We introduce Graph-guided Ordering with Local Refinement (GO-LR), show its equivalence to weighted Minimum Linear Arrangement, and interpret the practical solver as a TSP-path-style surrogate. We propose GOTabPFN,which builds on GO-LR, and a Neuro-Inspired Subunit Compression (NSC) unit to pool locally adjacent ordered features into meta-features, yielding a compact representation that makes TabPFN-style prediction practical in HDLSS regimes. Across tabular benchmarks, GOTabPFN improves stability and accuracy under tight token budgets.
Accepted to the 43rd International Conference on Machine Learning (ICML 2026). Code and resources are available at: GitHub: this https URL PyPI: this https URL Project webpage: this https URL Hugging Face Space: this https URL and this https URL
Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Machine Learning (stat.ML)
arXiv:2606.05441 [cs.LG]
(or arXiv:2606.05441v1 [cs.LG] for this version)
https://doi.org/10.48550/arXiv.2606.05441 arXiv-issued DOI via DataCite (pending registration)
Full text under the publisher's open licence · 155 words
Related coverage
GOTabPFN: From Feature Ordering to Compact Tokenization for Tabular Foundation Models on High-Dimensional Data
arXiv · Jun 9, 2026
Build Your Own Transaction Foundation Model for Financial Intelligence
NVIDIA · Jun 16, 2026
SOMTab: Set-Order Mamba for Efficient Tabular In-Context Learning
arXiv · Sep 1, 2026
Links #5: 2026/07
LessWrong · Jul 31, 2026