SCOPE-FE: Structured Control of Operator and Pairwise Exploration for Feature Engineering via Quality-Aware Candidate-Space Reduction

Abstract:Automatic feature engineering can improve predictive performance on tabular data by generating diverse feature transformations. However, the candidate space induced by combinations of input features and operators grows rapidly with dimensionality, resulting in substantial computational cost. We propose SCOPE-FE, a framework that controls the search space before candidate generation. SCOPE-FE combines FeatureClustering, a structural pair gate based on mixed-type feature association, with OperatorProbing, a dataset-specific utility control over operators. Unlike conventional expand-and-reduce approaches that generate a large candidate set and prune it afterward, SCOPE-FE focuses computation on a smaller, data-dependent candidate pool. Across ten OpenFE benchmark datasets, SCOPE-FE achieves a median candidate-space reduction of 82.9% and lowers component-summed feature-engineering time-including separately measured FeatureClustering overhead-on all ten datasets, yielding a geometric-mean speedup of 2.66x and a maximum speedup of 5.48x. Despite this reduction, SCOPE-FE is within the stated practical-equivalence margin of OpenFE on 8 of 10 datasets. An exhaustive candidate audit shows enrichment above uniform-random expectation on 8 of 10 datasets, with a median enrichment of 1.35x. Against Random-Pair, SCOPE-FE has higher enrichment on 6 of 10 datasets, with 5 of 10 significant; against Random-Operator and Random-Joint, it has higher enrichment on 8 of 10 datasets, with 8 of 10 significant for each. These results demonstrate that pre-generation search-space control can substantially reduce feature-engineering time while retaining a utility-enriched candidate pool under the OpenFE-compatible evaluation protocol.

Substantially revised version. FeatureClustering was changed from fuzzy soft clustering to deterministic agglomerative hard clustering. The methodology and experiments were extensively updated, including exhaustive candidate audits, budget-matched randomization controls, component ablations, probe-seed sensitivity, and intra-vs-inter cluster analysis

Machine Learning (stat.ML); Machine Learning (cs.LG)

arXiv:2604.27025 [stat.ML]

(or arXiv:2604.27025v2 [stat.ML] for this version)