analiza aiqbench
This paper addresses the computational bottleneck of Tabular Foundation Models (TFMs) by introducing ARASH, a method that optimizes in-context learning through adaptive shot selection. By leveraging local neighborhood analysis to retrieve relevant training rows, the approach drastically reduces prompt length and memory usage while maintaining accuracy comparable to full fine-tuning models like TabPFN. This efficiency gain is critical for deploying large-scale tabular AI systems where resource constraints are significant.
ARASH: Adaptive Retrieval And Shot Selection for Tabular Prediction
arXiv:2608.17856v1 Announce Type: new Abstract: Tabular prediction is a critical task across numerous applications. The recent success of large language models has sparked various approaches for adapting them to the tabular domain. A prevalent strategy involves training or fine-tuning specialized Tabular Foundation Models (TFMs) such as TabPFN. However, TFMs require substantial computational resources, and frequent model retraining is often impractical. In-context learning (ICL), specifically, few-shot prompting, offers a resource-efficient alternative to enhance performance. Yet, identifying the most relevant rows to serve as shots remains a challenge for tabular data. This paper introduces ARASH (Adaptive, query-specific Retrieval And Shot selection), a method that improves TFM efficiency by selecting optimal shots based on local neighborhood analysis within the training set. Our results demonstrate that ARASH reduces the prompt length and memory usage of TabPFN by 1261.5$\times$ and 2.56$\times$, respectively, while providing comparable accuracy.
Abstract:Tabular prediction is a critical task across numerous applications. The recent success of large language models has sparked various approaches for adapting them to the tabular domain. A prevalent strategy involves training or fine-tuning specialized Tabular Foundation Models (TFMs) such as TabPFN. However, TFMs require substantial computational resources, and frequent model retraining is often impractical. In-context learning (ICL), specifically, few-shot prompting, offers a resource-efficient alternative to enhance performance. Yet, identifying the most relevant rows to serve as shots remains a challenge for tabular data. This paper introduces ARASH (Adaptive, query-specific Retrieval And Shot selection), a method that improves TFM efficiency by selecting optimal shots based on local neighborhood analysis within the training set. Our results demonstrate that ARASH reduces the prompt length and memory usage of TabPFN by 1261.5$\times$ and 2.56$\times$, respectively, while providing comparable accuracy.
Artificial Intelligence (cs.AI)
arXiv:2608.17856 [cs.AI]
(or arXiv:2608.17856v1 [cs.AI] for this version)
https://doi.org/10.48550/arXiv.2608.17856 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 230 cuvinte
Subiecte conexe
Test-Time Compute for Tabular Foundation Models: Mechanisms, Gains, and Limits
arXiv · 9 oct. 2026
Kernel-Based Steering of CLIP with Vision-Language Model Preferences
arXiv · 30 sept. 2026
When Less Compute Is More: Adaptive Early Exit Improves Pretrained Outlier Detection
arXiv · 30 sept. 2026
What's Continual Learning, and Why Might We Expect To See It In Advanced LLM Agents?
LessWrong · 12 iun. 2026