aiqbench analysis
POSSE-kNN introduces a novel ensemble approach for nearest neighbor classification by integrating pathwise selection, bootstrap sampling, and out-of-bag error screening to better align neighborhoods with local class geometry. The study demonstrates improved aggregate accuracy and calibration on binary tabular benchmarks compared to standard k-NN, SVMs, and other established methods. This work advances practical machine learning techniques specifically for handling complex data distributions where fixed query-centered approaches underperform.
POSSE-kNN: Pathwise Out-of-Bag Selected Subspace Ensembles for Binary Classification
arXiv:2211.11278v3 Announce Type: replace-cross Abstract: Nearest neighbour classification is attractive for tabular data, but its performance can deteriorate when a fixed query centred neighbourhood does not follow the local class geometry. This study evaluates POSSE-$k$NN, a pathwise $k$ nearest neighbour ensemble that combines bootstrap sampling, random feature subspaces, out-of-bag (OOB) screening, and selective voting. Within each randomized candidate, pathwise selection first chooses the training observation nearest to the query and then chooses each subsequent neighbour relative to the observation accepted at the preceding step. After the candidate is fully specified, its OOB error is computed. Five hundred candidates are generated, ranked by OOB error, and the best 25% are retained. The method is evaluated on ten binary benchmark datasets using repeated 70/30 train/test partitions and six established comparators. Across the dataset level means, POSSE-kNN attains an accuracy of 0.740, Cohen's kappa of 0.412, and a Brier score of 0.175, giving the best aggregate result for all three criteria. It has the highest unrounded mean accuracy and kappa on eight datasets; RkNN and SVM lead on the other two. A neighbourhood size analysis on three datasets shows stable behaviour for k={3, 5, 7} when the path geometry is informative and identifies a case in which alternative neighbourhood rules are preferable.
Abstract:Nearest neighbour classification is attractive for tabular data, but its performance can deteriorate when a fixed query centred neighbourhood does not follow the local class geometry. This study evaluates POSSE-$k$NN, a pathwise $k$ nearest neighbour ensemble that combines bootstrap sampling, random feature subspaces, out-of-bag (OOB) screening, and selective voting. Within each randomized candidate, pathwise selection first chooses the training observation nearest to the query and then chooses each subsequent neighbour relative to the observation accepted at the preceding step. After the candidate is fully specified, its OOB error is computed. Five hundred candidates are generated, ranked by OOB error, and the best 25% are retained. The method is evaluated on ten binary benchmark datasets using repeated 70/30 train/test partitions and six established comparators. Across the dataset level means, POSSE-kNN attains an accuracy of 0.740, Cohen's kappa of 0.412, and a Brier score of 0.175, giving the best aggregate result for all three criteria. It has the highest unrounded mean accuracy and kappa on eight datasets; RkNN and SVM lead on the other two. A neighbourhood size analysis on three datasets shows stable behaviour for k={3, 5, 7} when the path geometry is informative and identifies a case in which alternative neighbourhood rules are preferable.
Machine Learning (stat.ML); Machine Learning (cs.LG)
arXiv:2211.11278 [stat.ML]
(or arXiv:2211.11278v3 [stat.ML] for this version)
https://doi.org/10.48550/arXiv.2211.11278 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 299 words
Related coverage
TeamHerald@CHIPSAL 2026: Hate Speech Detection and Sentiment Analysis of Nepali Memes using Transformer-based Architectures and Ensemble Learning
arXiv · Jun 9, 2026
Fine-Grained Emotion Classification from Mobile App Reviews: An Empirical Study with Large Language Models
arXiv · Oct 6, 2026
Beyond Rebalancing: Benchmarking Binary Classifiers Under Class Imbalance Without Rebalancing Techniques
arXiv · Jun 16, 2026
Hierarchical Specialised Ensembles for Classification of Zebrafish Phenotypes Using the Selected Image Recognition Methods
arXiv · Jul 20, 2026