analiza aiqbench
This paper analyses the proportion-of-positive-values (PPV) pooling operator used in ROCKET-family time series classifiers, proving translation invariance under periodic boundaries and bounding sensitivity for a smooth surrogate. Its experiments on the UCR archive suggest that more kernels yield diminishing returns and that longer kernels help representation but not always test accuracy, so richer features do not automatically generalise better. For ML readers, it offers a theoretical account of why simple fixed-kernel pipelines work, though the results are scoped to time series classification and the finite-boundary effects remain only partially characterised.
A Mechanistic Interpretation of Prescribed-Parameter Convolutional Kernels
arXiv:2502.17061v2 Announce Type: replace-cross Abstract: Prescribed-parameter convolutional transforms achieve strong time series classification performance using fixed kernels, simple pooling operations, and a linear classifier. Their success raises a mechanistic question: which properties of the representation make this construction effective? We study the proportion of positive values (PPV), a pooling operator shared by ROCKET, MiniROCKET, and MultiROCKET. We establish exact invariance to circular translations under periodic boundaries and quantify departures from invariance under finite boundaries. A complementary analysis bounds the sensitivity of a smooth PPV surrogate. We then define the population pairwise geometry induced by a distribution of convolutional parameters and show how a finite random kernel bank concentrates around that geometry. This separates the role of the parameter distribution from the number of sampled kernels. Experiments on the UCR classification collection examine effective dimensionality, distance correlation with the input, and classification accuracy using PPV-only, unit-energy ROCKET kernels. Increasing kernel count produces diminishing representational gains. Increasing kernel length generally raises effective dimensionality and distance correlation, while the best test performance occurs at intermediate lengths and training accuracy remains close to perfect. Together, these results support an interpretation in which nonlinear pooling creates a richer but structured representation that can favour linear separation, while further representational richness need not improve generalisation.
Abstract:Prescribed-parameter convolutional transforms achieve strong time series classification performance using fixed kernels, simple pooling operations, and a linear classifier. Their success raises a mechanistic question: which properties of the representation make this construction effective? We study the proportion of positive values (PPV), a pooling operator shared by ROCKET, MiniROCKET, and MultiROCKET. We establish exact invariance to circular translations under periodic boundaries and quantify departures from invariance under finite boundaries. A complementary analysis bounds the sensitivity of a smooth PPV surrogate. We then define the population pairwise geometry induced by a distribution of convolutional parameters and show how a finite random kernel bank concentrates around that geometry. This separates the role of the parameter distribution from the number of sampled kernels. Experiments on the UCR classification collection examine effective dimensionality, distance correlation with the input, and classification accuracy using PPV-only, unit-energy ROCKET kernels. Increasing kernel count produces diminishing representational gains. Increasing kernel length generally raises effective dimensionality and distance correlation, while the best test performance occurs at intermediate lengths and training accuracy remains close to perfect. Together, these results support an interpretation in which nonlinear pooling creates a richer but structured representation that can favour linear separation, while further representational richness need not improve generalisation.
17 pages, 3 figures. Substantially revised version with a new title. Replaces the sparsity-based interpretation with an analysis of PPV representations through translation invariance, concentration of random-feature geometry, and expanded experiments on kernel number and length
Machine Learning (stat.ML); Machine Learning (cs.LG)
arXiv:2502.17061 [stat.ML]
(or arXiv:2502.17061v2 [stat.ML] for this version)
Text integral sub licența deschisă a publicației · 254 cuvinte