From Uniform to Learned Knots: A Study of Spline-Based Numerical Encodings for Tabular Deep Learning

Abstract:Numerical preprocessing remains a critical component of tabular deep learning, as the representation of continuous features can strongly affect downstream performance. We systematically study spline-based numerical encodings, including B-splines, M-splines, and integrated splines (I-splines), under uniform, quantile-based, target-aware, and learnable-knot placement. For the learnable variants, we adopt a differentiable knot parameterization that enables stable end-to-end optimization of knot locations jointly with the backbone. We evaluate these encodings on a diverse collection of public regression and classification datasets using MLP, ResNet, and FT-Transformer backbones, and compare them against common numerical preprocessing baselines. Our results show that the effectiveness of numerical encoding depends strongly on the task, encoding size, and backbone. For classification, piecewise-linear encoding (PLE) is the most robust choice overall, while spline-based encodings remain competitive. For regression, no single encoding dominates, with performance depending on the spline family and knot-placement strategy, and larger gains generally observed for MLP and ResNet than for FT-Transformer. Learnable-knot variants can be optimized stably but may substantially increase training cost. Overall, numerical encodings should therefore be assessed jointly in terms of predictive performance and computational overhead. The implementation is publicly available at this https URL.

Machine Learning (cs.LG)

arXiv:2604.05635 [cs.LG]

(or arXiv:2604.05635v2 [cs.LG] for this version)

https://doi.org/10.48550/arXiv.2604.05635 Focus to learn more arXiv-issued DOI via DataCite

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.