analiza aiqbench
Tydra addresses the computational bottleneck in transformer-based tabular models by integrating State Space Models to achieve subquadratic inference speeds without sacrificing predictive accuracy. This hybrid architecture offers a practical pathway for deploying foundation models on larger datasets where efficiency is critical, bridging the gap between high performance and resource constraints.
Tydra: An Efficient Hybrid Model for Tabular Data
arXiv:2608.21199v1 Announce Type: new Abstract: Transformer-based tabular foundation models such as TabPFN achieve strong predictive performance but incur quadratic computational cost with context length. On the other hand, subquadratic SSM-based alternatives such as Hydra trade away accuracy for efficiency. To balance both, we introduce Tydra, a hybrid Transformer-State Space Model (SSM) architecture for tabular in-context learning that interleaves attention and SSM layers. Across 30 OpenML datasets, Tydra reduces inference time by 30% relative to TabPFN while retaining much of its predictive performance. Tydra also outperforms an approximately ten-times-larger Hydra model while providing faster inference. The results indicate that hybrid architectures are a promising direction for tabular foundation models.
Abstract:Transformer-based tabular foundation models such as TabPFN achieve strong predictive performance but incur quadratic computational cost with context length. On the other hand, subquadratic SSM-based alternatives such as Hydra trade away accuracy for efficiency. To balance both, we introduce Tydra, a hybrid Transformer-State Space Model (SSM) architecture for tabular in-context learning that interleaves attention and SSM layers. Across 30 OpenML datasets, Tydra reduces inference time by 30% relative to TabPFN while retaining much of its predictive performance. Tydra also outperforms an approximately ten-times-larger Hydra model while providing faster inference. The results indicate that hybrid architectures are a promising direction for tabular foundation models.
Machine Learning (cs.LG)
arXiv:2608.21199 [cs.LG]
(or arXiv:2608.21199v1 [cs.LG] for this version)
https://doi.org/10.48550/arXiv.2608.21199 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 197 cuvinte
Subiecte conexe
Quantum Circuits Cut Data Imbalance Recall Gap by 10–25%
Quantum Zeitgeist · 13 iul. 2026
Introducing TabFM: A zero-shot foundation model for tabular data
Google Research · 30 iun. 2026
Quantum Circuits Synthesise Realistic Fraud Data to Beat Detection Bias
Quantum Zeitgeist · 23 mai 2026
Technical Comparative Benchmarking Study: Advanced AI Hybrid Methods for Renewable Energy Farm Optimization and Forecasting
arXiv · 28 aug. 2026