analiza aiqbench
This paper introduces "effective boundary complexity," a diagnostic metric that quantifies the cost of scarce labels in ambiguous regions near decision boundaries for frozen representations. By normalizing mass around the learned boundary, it provides a rigorous tool for estimating label budgets and optimizing linear probes without requiring additional training data. The method demonstrates strong predictive power across controlled transformations and real-world datasets like CIFAR-100, offering a practical solution for low-label transfer scenarios.
Boundary Mass in Feature Space as a Label-Budget Diagnostic for Representations
De la arXiv
Text integralarXiv:2611.11525v1 Announce Type: new Abstract: Low-label transfer is often decided by a practical question: after a representation looks useful, how many labels are still needed to learn the downstream boundary? Transferability and probe-quality scores characterize global alignment, likelihood, margins, and calibration. Boundary-near target mass complements these diagnostics by measuring the ambiguous regions where scarce labels are most costly. We propose effective boundary complexity, a label-budget diagnostic for frozen representations and linear probes.
It measures mass in a thin strip around the learned boundary and normalizes it by class balance, so dense ambiguous regions and predicted-class imbalance are both counted. We prove that, under a smooth representation map, density change and tangential surface change cancel at first order, leaving boundary-normal stretch or compression as the governing factor. This yields a cached-feature estimator that fits a probe on one split and counts independent held-out features near its boundary.
Experiments support the diagnostic at three levels. Controlled transformations match the predicted law across 74 maps with 0.38% median relative error. On real frozen features, normal-direction changes move both measured boundary mass and label demand, while tangent controls preserve the first-order coefficient. A differentiable surrogate reduces measured complexity by 16-36% at matched top-1 utility. Finally, CIFAR-100 and Tiny-ImageNet experiments support boundary mass as a second-stage diagnostic for near-tied representations. Five-way experiments show strong OvR-interface backbone-ranking agreement (Spearman 0.872), while acquisition experiments show high correlations between initial boundary mass and realized label budgets across three sampling policies (0.779-0.830). Code is available at https://github.com/qqaqq-code/boundary_mass.
Abstract:Low-label transfer is often decided by a practical question: after a representation looks useful, how many labels are still needed to learn the downstream boundary? Transferability and probe-quality scores characterize global alignment, likelihood, margins, and calibration. Boundary-near target mass complements these diagnostics by measuring the ambiguous regions where scarce labels are most costly. We propose effective boundary complexity, a label-budget diagnostic for frozen representations and linear probes. It measures mass in a thin strip around the learned boundary and normalizes it by class balance, so dense ambiguous regions and predicted-class imbalance are both counted.
We prove that, under a smooth representation map, density change and tangential surface change cancel at first order, leaving boundary-normal stretch or compression as the governing factor. This yields a cached-feature estimator that fits a probe on one split and counts independent held-out features near its boundary. Experiments support the diagnostic at three levels. Controlled transformations match the predicted law across 74 maps with 0.38% median relative error. On real frozen features, normal-direction changes move both measured boundary mass and label demand, while tangent controls preserve the first-order coefficient.
A differentiable surrogate reduces measured complexity by 16-36% at matched top-1 utility. Finally, CIFAR-100 and Tiny-ImageNet experiments support boundary mass as a second-stage diagnostic for near-tied representations. Five-way experiments show strong OvR-interface backbone-ranking agreement (Spearman 0.872), while acquisition experiments show high correlations between initial boundary mass and realized label budgets across three sampling policies (0.779-0.830). Code is available at this https URL.
Machine Learning (cs.LG); Artificial Intelligence (cs.AI)
arXiv:2611.11525 [cs.LG]
(or arXiv:2611.11525v1 [cs.LG] for this version)
https://doi.org/10.48550/arXiv.2611.11525 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 342 cuvinte