aiqbench analysis
This paper introduces a directional linear separability measure (LSM) that provides a finite-sample diagnostic for the geometric structure of neural representations beyond standard predictive metrics. By establishing invariance under linear embeddings and characterizing class-wise intrusion, the method effectively isolates information loss from mere linear reparameterization effects. This geometric perspective offers a rigorous tool for analyzing how specific deep-learning components and nonlinearities impact the affine separability of learned features.
A Geometric Measure of Linear Separability for Neural Representations
arXiv:2606.08721v1 Announce Type: new Abstract: Modern neural classifiers commonly rely on linear readouts, yet predictive metrics alone do not characterize the class-wise geometry of the representations on which such readouts operate. We introduce the directional linear separability measure (LSM), a finite-sample diagnostic for one-sided affine separability. For a target class A and a competing set B, LSM searches over affine halfspaces that contain all samples in A and measures the smallest competing-sample intrusion that must remain on the target side, normalized by |A|. The resulting quantity is asymmetric, class-wise, target-normalized, and applicable to finite representations extracted from neural networks. We establish its supporting-hyperplane characterization, relate it to optimal affine classification accuracy, and prove invariance under full-rank linear embeddings. These results separate changes caused by linear reparameterization from those caused by information loss or nonlinear geometric transformations. We also give a penalty-based affine search for estimating class-wise LSM in high-dimensional features, with reported values computed from the original discrete preservation and violation criterion. Finally, we analyze coordinatewise gated nonlinearities as finite-sample geometric operators and empirically use LSM to diagnose class-wise intrusion across common deep-learning components and architectures.
Abstract:Modern neural classifiers commonly rely on linear readouts, yet predictive metrics alone do not characterize the class-wise geometry of the representations on which such readouts operate. We introduce the directional linear separability measure (LSM), a finite-sample diagnostic for one-sided affine separability. For a target class A and a competing set B, LSM searches over affine halfspaces that contain all samples in A and measures the smallest competing-sample intrusion that must remain on the target side, normalized by |A|. The resulting quantity is asymmetric, class-wise, target-normalized, and applicable to finite representations extracted from neural networks. We establish its supporting-hyperplane characterization, relate it to optimal affine classification accuracy, and prove invariance under full-rank linear embeddings. These results separate changes caused by linear reparameterization from those caused by information loss or nonlinear geometric transformations. We also give a penalty-based affine search for estimating class-wise LSM in high-dimensional features, with reported values computed from the original discrete preservation and violation criterion. Finally, we analyze coordinatewise gated nonlinearities as finite-sample geometric operators and empirically use LSM to diagnose class-wise intrusion across common deep-learning components and architectures.
Machine Learning (cs.LG)
arXiv:2606.08721 [cs.LG]
(or arXiv:2606.08721v1 [cs.LG] for this version)
https://doi.org/10.48550/arXiv.2606.08721 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 275 words
Related coverage
Stochastic Separability of Embedding Manifolds
arXiv · Aug 25, 2026
How do neural networks regress cubic polynomials? Apparently, they use a trick invented in Milan 500 years ago
LessWrong · Jul 22, 2026
Representation of syntax in LLMs through the lens of linear distance and similarity-aware entropy
arXiv · Aug 31, 2026
PIRAMID: Progress and Plans
LessWrong · Jul 27, 2026