analiza aiqbench
This research introduces Selective Edge Masking, a novel mechanism that leverages Optimal Brain Damage theory to prune redundant connections in fully connected layers and mitigate performance degradation from noisy labels. By restricting the backpropagation pathways of erroneous gradients, this approach offers a dynamic architectural adaptation strategy that complements existing robust loss functions and sample selection techniques. The method is validated as a plug-and-play component capable of enhancing noise robustness across various models, including emerging architectures like Kolmogorov-Arnold Networks.
Label-Noise Resistant Learning via Optimal Brain Damage Masking
arXiv:2508.09697v4 Announce Type: replace Abstract: Noisy labels are inevitable in real-world multimedia applications. Due to the strong memorization capacity of deep neural networks, these noisy labels cause significant performance degradation. Existing noise-robust methods have mainly focused on robust loss functions and sample selection strategies, with comparatively limited exploration of dynamic architectural adaptation. In this paper, we rethink the role of classifier connectivity under label noise. Intuitively, performance degradation stems from the backpropagation of noisy gradients. Since the final classifier layer acts as the primary gateway for this error propagation, selectively discarding redundant connections can restrict the backpropagation pathways of noisy gradients. Consequently, to identify redundant connections, we leverage the seminal Optimal Brain Damage (OBD) theory from model compression, which posits that parameters causing negligible loss perturbation can be removed. Guided by this principle, we show that masking low-activation edges limits the estimated loss perturbation to preserve the model's fitting capacity, while tightening a derived upper bound on noise-induced gradient error. To bridge this theoretical insight with practical training, we propose a novel Selective Edge Masking (SEM) mechanism for the fully connected (FC) layer to enhance noise robustness. It can adaptively retain critical edges for information propagation while suppressing gradient errors caused by noisy labels. As a plug-and-play component, SEM can be seamlessly integrated into various noise-robust methods. Additionally, we validate SEM's applicability by applying it to the newly emerged Kolmogorov-Arnold Network (KAN) employed as a classifier. Extensive evaluations on synthetic and real-world benchmarks demonstrate that our approach achieves state-of-the-art performance.
Abstract:Noisy labels are inevitable in real-world multimedia applications. Due to the strong memorization capacity of deep neural networks, these noisy labels cause significant performance degradation. Existing noise-robust methods have mainly focused on robust loss functions and sample selection strategies, with comparatively limited exploration of dynamic architectural adaptation. In this paper, we rethink the role of classifier connectivity under label noise. Intuitively, performance degradation stems from the backpropagation of noisy gradients. Since the final classifier layer acts as the primary gateway for this error propagation, selectively discarding redundant connections can restrict the backpropagation pathways of noisy gradients. Consequently, to identify redundant connections, we leverage the seminal Optimal Brain Damage (OBD) theory from model compression, which posits that parameters causing negligible loss perturbation can be removed. Guided by this principle, we show that masking low-activation edges limits the estimated loss perturbation to preserve the model's fitting capacity, while tightening a derived upper bound on noise-induced gradient error. To bridge this theoretical insight with practical training, we propose a novel Selective Edge Masking (SEM) mechanism for the fully connected (FC) layer to enhance noise robustness. It can adaptively retain critical edges for information propagation while suppressing gradient errors caused by noisy labels. As a plug-and-play component, SEM can be seamlessly integrated into various noise-robust methods. Additionally, we validate SEM's applicability by applying it to the newly emerged Kolmogorov-Arnold Network (KAN) employed as a classifier. Extensive evaluations on synthetic and real-world benchmarks demonstrate that our approach achieves state-of-the-art performance.
Machine Learning (cs.LG); Computer Vision and Pattern Recognition (cs.CV)
arXiv:2508.09697 [cs.LG]
(or arXiv:2508.09697v4 [cs.LG] for this version)
Text integral sub licența deschisă a publicației · 262 cuvinte
Subiecte conexe
Humanity’s Last Invention — Richard Socher of Recursive
Latent Space · 14 sept. 2026
Reviewing failure modes in science as we prepare for AI researchers
LessWrong · 8 oct. 2026
Some Intuitions on Steering Vectors
LessWrong · 29 sept. 2026
The J-Space Debate, Agent Swarms, and Pacing Frontier AI - Digital Minds Newsletter #4
LessWrong · 18 sept. 2026