aiqbench analysis
This research introduces Stochastic Sequential Search (SSS), specifically sSFFS, which overcomes the computational bottlenecks of traditional feature selection methods in very-high-dimensional datasets by replacing exhaustive sweeps with temperature-controlled sampling. The method maintains high criterion values and model accuracy even at dimensions exceeding 10,000 while reducing evaluation costs significantly compared to uniform sampling or standard ranking approaches. This advancement is critical for scaling machine learning pipelines where computational budget limits the feasibility of rigorous feature selection in massive data environments.
Stochastic Sequential Search in Very-High-Dimensional Feature Selection
arXiv:2608.01502v1 Announce Type: new Abstract: Sequential subset search -- forward selection with floating backtracking and its descendants -- remains the quality reference in feature selection, but every member of the family sweeps the full pool of remaining candidate features at each step, which excludes it from very-high-dimensional problems; there, only individual-feature ranking remains practical, and it models feature interplay weakly or not at all. We introduce a budgeted sampled step operator pair that replaces the full sweeps by a fixed number of candidate evaluations per step. Candidates are drawn by temperature-controlled softmax sampling from dependency-aware per-feature statistics learned online from every criterion evaluation the search performs, guarded by a uniform exploration floor; per-step cost becomes independent of dimensionality. Substituting the operators turns any sequential method into its stochastic counterpart, defining the Stochastic Sequential Search (SSS) family; we study the stochastic counterpart of floating search, sSFFS. On 500-dimensional madelon, sSFFS retains at least 97% of the full-SFFS criterion value at every subset size at about a quarter of its evaluations, while uniform sampling at the same budget collapses on madelon's synergistic features. On 5,000-dimensional gisette, far beyond full-SFFS reach, sSFFS exceeds the saturated criterion level of DAF and BIF ranking at matched budgets; holdout validation shows that at 500 training samples the binding constraint beyond the sequential frontier becomes the criterion, not the search. On 10,105-dimensional reuters, under a trustworthy multinomial filter criterion, sSFFS dominates BIF and DAF on the search objective and on holdout accuracy at every subset size, in about two minutes of single-core evaluation work. A verified standalone implementation accompanies the paper.
Abstract:Sequential subset search -- forward selection with floating backtracking and its descendants -- remains the quality reference in feature selection, but every member of the family sweeps the full pool of remaining candidate features at each step, which excludes it from very-high-dimensional problems; there, only individual-feature ranking remains practical, and it models feature interplay weakly or not at all. We introduce a budgeted sampled step operator pair that replaces the full sweeps by a fixed number of candidate evaluations per step. Candidates are drawn by temperature-controlled softmax sampling from dependency-aware per-feature statistics learned online from every criterion evaluation the search performs, guarded by a uniform exploration floor; per-step cost becomes independent of dimensionality. Substituting the operators turns any sequential method into its stochastic counterpart, defining the Stochastic Sequential Search (SSS) family; we study the stochastic counterpart of floating search, sSFFS. On 500-dimensional madelon, sSFFS retains at least 97% of the full-SFFS criterion value at every subset size at about a quarter of its evaluations, while uniform sampling at the same budget collapses on madelon's synergistic features. On 5,000-dimensional gisette, far beyond full-SFFS reach, sSFFS exceeds the saturated criterion level of DAF and BIF ranking at matched budgets; holdout validation shows that at 500 training samples the binding constraint beyond the sequential frontier becomes the criterion, not the search. On 10,105-dimensional reuters, under a trustworthy multinomial filter criterion, sSFFS dominates BIF and DAF on the search objective and on holdout accuracy at every subset size, in about two minutes of single-core evaluation work. A verified standalone implementation accompanies the paper.
23 pages, 5 figures, 2 tables. Ancillary files: a standalone implementation of sSFFS (MIT license, verified bit-identical to the reference implementation), a checksum-verified converter building the exact experiment datasets from the UCI originals, the derived Reuters-21578 term matrix used in the experiments (research use only), and the configuration files of all experiments
Machine Learning (cs.LG); Computer Vision and Pattern Recognition (cs.CV); Machine Learning (stat.ML)
arXiv:2608.01502 [cs.LG]
(or arXiv:2608.01502v1 [cs.LG] for this version)
https://doi.org/10.48550/arXiv.2608.01502 arXiv-issued DOI via DataCite (pending registration)
Full text under the publisher's open licence · 338 words
Related coverage
$L_0$ Feature Selection via Monte Carlo Tree Search in High-Dimensional Financial Regression
arXiv · Oct 12, 2026
Selecting a vector store for Amazon Bedrock Knowledge Bases
AWS · Sep 17, 2026
The Two-Century Path That Made Quantum Computing Inevitable
Quantum Computing Report · Sep 28, 2026
Agentic High-Dimensional Bayesian Optimization with Hypothesis- and Evidence-Guided Search
arXiv · Sep 30, 2026