DASH: Dual-Branch Score Distillation for Guidance-Calibrated Compact Diffusion Models

Abstract:Parameter compression of class-conditional diffusion models exposes a structural limitation in output-level distillation: supervising only the guided output leaves the two score branches non-identifiable, so the classifier-free guidance gap is not determined in the student. The zero-loss set admits degenerate solutions in which both branches collapse toward identical predictions, and guidance loses its effect at inference despite low training loss. For each fixed input the objective is exactly flat in a branch-output direction at every residual value, so the ambiguity is a property of the output-level objective rather than a poor local minimum. This paper introduces DASH, which supervises the conditional and unconditional branches independently. An anchor term regularises the conditional prediction toward ground-truth noise, and the teacher's final learned per-timestep curriculum transfers into the student as a frozen prior. Across CIFAR-10, CIFAR-100, and class-conditional ImageNet-64, a more than 5x compressed student stays within four FID points of its teacher and recovers at least 89% of the teacher's guidance-gap magnitude, where no other two-branch baseline with defined calibration exceeds 82%. Ablation isolates unconditional supervision as the term that separates this formulation from output-level distillation. The account is quantitative: the null space fixes the ratio between a composite student's two branch errors, and the measured ratios approximately match that prediction on all three datasets.

30 pages, 12 figures, 19 tables. Code: this https URL

Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Machine Learning (cs.LG)

arXiv:2606.00798 [cs.CV]

(or arXiv:2606.00798v2 [cs.CV] for this version)

https://doi.org/10.48550/arXiv.2606.00798 Focus to learn more arXiv-issued DOI via DataCite

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.