analiza aiqbench
This paper proposes IW-CFM, a method that trains a generative model to target a distribution powered by a guidance exponent, rather than relying on classifier-free guidance to approximate it. The authors report that standard CFG at matched scale can visibly miss the analytical target on synthetic data, and that a lightweight corrector improves alignment on ImageNet. For practitioners, the key questions are how well the self-normalized, regularized weight estimates hold up at larger scales and beyond class-conditional ImageNet, and whether the gains persist against stronger guidance baselines.
Learning $\omega$-Powered Conditional Distributions with Importance-Weighted Flow Matching
arXiv:2610.16419v1 Announce Type: new Abstract: Classifier-Free Guidance (CFG) with guidance scale $\gamma$ is commonly motivated by the $\omega$-powered conditional distribution obtained by setting $\omega=\gamma$, but CFG does not, in general, sample from this distribution exactly. We study whether the powered target can instead be learned explicitly. We introduce Importance-Weighted Conditional Flow Matching (IW-CFM), which reweights conditional flow-matching examples using density ratios obtained from frozen conditional and unconditional flow models. With exact normalized weights, the resulting objective has the same gradient as flow matching on the powered target; in practice, we use self-normalized estimation and a regularized empirical approximation for high-dimensional data. On synthetic distributions, IW-CFM recovers analytical powered targets for which numerically matched CFG, $\gamma=\omega$, exhibits a visible mismatch. On ImageNet, an amortized lightweight corrector achieves better alignment than fixed- and matched-guidance CFG with the corresponding weighted and filtered reference distributions. We additionally present an exploratory CLIP analysis of powered-density rankings on real ImageNet data. Aggregate shifts across selected prompt-specific contrasts coexist with heterogeneous classwise responses, including nonmonotonic trajectories. These results provide a practical route for studying and approximating model-defined powered conditional distributions.
Abstract:Classifier-Free Guidance (CFG) with guidance scale $\gamma$ is commonly motivated by the $\omega$-powered conditional distribution obtained by setting $\omega=\gamma$, but CFG does not, in general, sample from this distribution exactly. We study whether the powered target can instead be learned explicitly. We introduce Importance-Weighted Conditional Flow Matching (IW-CFM), which reweights conditional flow-matching examples using density ratios obtained from frozen conditional and unconditional flow models. With exact normalized weights, the resulting objective has the same gradient as flow matching on the powered target; in practice, we use self-normalized estimation and a regularized empirical approximation for high-dimensional data. On synthetic distributions, IW-CFM recovers analytical powered targets for which numerically matched CFG, $\gamma=\omega$, exhibits a visible mismatch. On ImageNet, an amortized lightweight corrector achieves better alignment than fixed- and matched-guidance CFG with the corresponding weighted and filtered reference distributions. We additionally present an exploratory CLIP analysis of powered-density rankings on real ImageNet data. Aggregate shifts across selected prompt-specific contrasts coexist with heterogeneous classwise responses, including nonmonotonic trajectories. These results provide a practical route for studying and approximating model-defined powered conditional distributions.
Machine Learning (cs.LG); Computer Vision and Pattern Recognition (cs.CV)
arXiv:2610.16419 [cs.LG]
(or arXiv:2610.16419v1 [cs.LG] for this version)
https://doi.org/10.48550/arXiv.2610.16419 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 278 cuvinte
Subiecte conexe
The Entangled Dimensions of Decision Theory
LessWrong · 30 iul. 2026
Deconfusing Corrigibility
AI Alignment Forum · 17 oct. 2026
What's Continual Learning, and Why Might We Expect To See It In Advanced LLM Agents?
LessWrong · 12 iun. 2026
Challenge: Hand coding weights for efficient sequence memorisation
AI Alignment Forum · 23 iul. 2026