analiza aiqbench
This paper is a machine learning study of model compression: it combines knowledge distillation with structured and unstructured pruning in an iterative loop to shrink a CNN for remote sensing classification. The reported gains are relative to one-shot pruning and single-stage distillation on EuroSAT and NWPU-RESISC45, and the claim of strong accuracy beyond 90% parameter reduction is the key number to check against the full tables. Readers working on edge deployment should watch whether the results hold on real embedded hardware, since parameter reduction does not always translate into latency or energy savings.
DAC-Loop: Distillation-Aware Compression for Remote Sensing CNN
arXiv:2610.35175v1 Announce Type: new Abstract: While large-scale models achieve strong results in remote sensing scene classification, deploying them on edge devices remains difficult due to limited compute, memory, and energy budgets. Compact CNN backbones are therefore a practical choice in many real-world settings, offering efficient inference with well-optimized implementations on embedded hardware. To push this efficiency further, we investigate how far a knowledge-distilled student model can be compressed while preserving high accuracy. We propose a Distillation-Aware Compression Loop (DAC-Loop) that integrates (i) knowledge distillation applied both before and after structured channel pruning, and (ii) an iterative compression schedule that alternates structured pruning with unstructured sparsification. Across EuroSAT and NWPU-RESISC45, the DAC-Loop consistently improves the accuracy-compression trade-off relative to standard one-shot pruning and single-stage KD recovery, maintaining strong accuracy even beyond 90% parameter reduction. These results demonstrate that combining distillation and pruning in a unified iterative loop offers an effective path toward compact remote sensing models.
Abstract:While large-scale models achieve strong results in remote sensing scene classification, deploying them on edge devices remains difficult due to limited compute, memory, and energy budgets. Compact CNN backbones are therefore a practical choice in many real-world settings, offering efficient inference with well-optimized implementations on embedded hardware. To push this efficiency further, we investigate how far a knowledge-distilled student model can be compressed while preserving high accuracy. We propose a Distillation-Aware Compression Loop (DAC-Loop) that integrates (i) knowledge distillation applied both before and after structured channel pruning, and (ii) an iterative compression schedule that alternates structured pruning with unstructured sparsification. Across EuroSAT and NWPU-RESISC45, the DAC-Loop consistently improves the accuracy-compression trade-off relative to standard one-shot pruning and single-stage KD recovery, maintaining strong accuracy even beyond 90% parameter reduction. These results demonstrate that combining distillation and pruning in a unified iterative loop offers an effective path toward compact remote sensing models.
Machine Learning (cs.LG)
arXiv:2610.35175 [cs.LG]
(or arXiv:2610.35175v1 [cs.LG] for this version)
https://doi.org/10.48550/arXiv.2610.35175 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
https://doi.org/10.1109/IGARSS60577.2026.11730294 Focus to learn more DOI(s) linking to related resources
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 254 cuvinte