aiqbench analysis
GOTabPFN addresses the High-Dimensional, Low-Sample Size (HDLSS) challenge by combining graph-guided feature ordering with a neuro-inspired compression unit to create compact representations for tabular data. This approach enables TabPFN-style prediction without retraining large backbones, significantly improving stability and accuracy under strict token budgets. The method is particularly relevant for practitioners needing efficient, high-performance models on datasets where sample size is limited relative to feature count.
GOTabPFN: From Feature Ordering to Compact Tokenization for Tabular Foundation Models on High-Dimensional Data
arXiv:2606.05441v2 Announce Type: replace Abstract: We investigate how to make small tabular foundation models effective for High-Dimensional, Low-Sample Size (HDLSS) tabular prediction without retraining large backbones. We introduce Graph-guided Ordering with Local Refinement (GO-LR), show its equivalence to weighted Minimum Linear Arrangement, and interpret the practical solver as a TSP-path-style surrogate. We propose GOTabPFN,which builds on GO-LR, and a Neuro-Inspired Subunit Compression (NSC) unit to pool locally adjacent ordered features into meta-features, yielding a compact representation that makes TabPFN-style prediction practical in HDLSS regimes. Across tabular benchmarks, GOTabPFN improves stability and accuracy under tight token budgets.
Abstract:We investigate how to make small tabular foundation models effective for High-Dimensional, Low-Sample Size (HDLSS) tabular prediction without retraining large backbones. We introduce Graph-guided Ordering with Local Refinement (GO-LR), show its equivalence to weighted Minimum Linear Arrangement, and interpret the practical solver as a TSP-path-style surrogate. We propose GOTabPFN,which builds on GO-LR, and a Neuro-Inspired Subunit Compression (NSC) unit to pool locally adjacent ordered features into meta-features, yielding a compact representation that makes TabPFN-style prediction practical in HDLSS regimes. Across tabular benchmarks, GOTabPFN improves stability and accuracy under tight token budgets.
Accepted to the 43rd International Conference on Machine Learning (ICML 2026). Code and resources GitHub this https URL PyPI this https URL Project webpage this https URL Hugging Face ZeroGPU this https URL CPU backup this https URL
Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Machine Learning (stat.ML)
arXiv:2606.05441 [cs.LG]
(or arXiv:2606.05441v2 [cs.LG] for this version)
Full text under the publisher's open licence · 146 words
Related coverage
GOTabPFN: From Feature Ordering to Compact Tokenization for Tabular Foundation Models on High-Dimensional Data
arXiv · Jun 6, 2026
Build Your Own Transaction Foundation Model for Financial Intelligence
NVIDIA · Jun 16, 2026
SOMTab: Set-Order Mamba for Efficient Tabular In-Context Learning
arXiv · Sep 1, 2026
Links #5: 2026/07
LessWrong · Jul 31, 2026