analiza aiqbench
This research introduces TREVIS, a method that leverages Transformer Variational Auto-Encoders to learn sparse decision trees through continuous latent space optimization. By replacing discrete search processes with gradient-based techniques, the approach successfully balances predictive accuracy with structural sparsity, addressing a key limitation in traditional tree-learning algorithms. The work advances machine learning interpretability by enabling the discovery of simpler models without sacrificing performance.
Learning Sparse Decision Trees via Transformer Variational Auto-Encoders
arXiv:2609.01430v1 Announce Type: new Abstract: Decision trees are among the most widely used models in machine learning, largely due to their transparent decision logic, making them well-suited for high-stakes decision-making contexts. However, most existing learning algorithms focus on predictive performance, overlooking the joint optimization of other desirable properties, such as structural sparsity. In this work we propose TREVIS, an approach for learning decision trees with respect to complex objectives, based on the exploration of the latent space of a Tree Transformer Variational Auto-Encoder (TTVAE). By mapping decision trees onto latent representations, TREVIS replaces the discrete search space with a continuous one, enabling gradient-based optimization via a differentiable surrogate model. We experiment with TREVIS for learning decision trees that jointly optimize predictive performance and sparsity. Results show that TREVIS discovers decision trees matching the predictive performance of existing near-optimal algorithms while improving their structural sparsity.
Abstract:Decision trees are among the most widely used models in machine learning, largely due to their transparent decision logic, making them well-suited for high-stakes decision-making contexts. However, most existing learning algorithms focus on predictive performance, overlooking the joint optimization of other desirable properties, such as structural sparsity. In this work we propose TREVIS, an approach for learning decision trees with respect to complex objectives, based on the exploration of the latent space of a Tree Transformer Variational Auto-Encoder (TTVAE). By mapping decision trees onto latent representations, TREVIS replaces the discrete search space with a continuous one, enabling gradient-based optimization via a differentiable surrogate model. We experiment with TREVIS for learning decision trees that jointly optimize predictive performance and sparsity. Results show that TREVIS discovers decision trees matching the predictive performance of existing near-optimal algorithms while improving their structural sparsity.
Machine Learning (cs.LG); Artificial Intelligence (cs.AI)
arXiv:2609.01430 [cs.LG]
(or arXiv:2609.01430v1 [cs.LG] for this version)
https://doi.org/10.48550/arXiv.2609.01430 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 236 cuvinte
Subiecte conexe
Lattice-Based Cryptography Explained
Quantum Zeitgeist · 15 aug. 2026
Dropless MoE Training in JAX with Primus-Turbo
AMD · 10 iun. 2026
Informe de riesgos de la IA de frontera (febrero–marzo de 2026)
METR · 19 mai 2026
Frontier AI Risk Report (February–March 2026)
METR · 19 mai 2026