analiza aiqbench
GILC introduces a plug-and-play framework that stabilizes controllable generation in discrete diffusion models by using gradient-informed logit correction without requiring retraining or fine-tuning. By employing a Jacobian-free mechanism, the method effectively handles high-dimensional discrete spaces while accommodating both differentiable and non-differentiable reward functions across tasks like protein sequence generation. This approach offers an efficient alternative to computationally expensive optimization methods for researchers working on generative AI models.
Plug-and-Play Guidance for Discrete Diffusion Models via Gradient-Informed Logit Correction
arXiv:2606.06303v2 Announce Type: replace-cross Abstract: Controllable generation with discrete diffusion models is often hindered by high computational overhead or the need for retraining. In this paper, we present \underline{\textbf{G}}radient-\underline{\textbf{I}}nformed \underline{\textbf{L}}ogit \underline{\textbf{C}}orrection (\textbf{GILC}), a plug-and-play framework that efficiently estimates guidance signals by repurposing the pretrained denoising network as a variational proxy. To circumvent the gradient instability inherent in high-dimensional discrete spaces, we introduce a Jacobian-free mechanism that directly corrects the clean prediction logits, facilitating stable and effective guidance. Our method accommodates both differentiable and non-differentiable reward functions. Extensive experiments across DNA, protein sequence, and molecular generation tasks demonstrate that GILC achieves state-of-the-art performance without additional training, frequently outperforming fine-tuning approaches.
Abstract:Controllable generation with discrete diffusion models is often hindered by high computational overhead or the need for retraining. In this paper, we present \underline{\textbf{G}}radient-\underline{\textbf{I}}nformed \underline{\textbf{L}}ogit \underline{\textbf{C}}orrection (\textbf{GILC}), a plug-and-play framework that efficiently estimates guidance signals by repurposing the pretrained denoising network as a variational proxy. To circumvent the gradient instability inherent in high-dimensional discrete spaces, we introduce a Jacobian-free mechanism that directly corrects the clean prediction logits, facilitating stable and effective guidance. Our method accommodates both differentiable and non-differentiable reward functions. Extensive experiments across DNA, protein sequence, and molecular generation tasks demonstrate that GILC achieves state-of-the-art performance without additional training, frequently outperforming fine-tuning approaches.
Machine Learning (cs.LG); Artificial Intelligence (cs.AI)
arXiv:2606.06303 [cs.LG]
(or arXiv:2606.06303v2 [cs.LG] for this version)
https://doi.org/10.48550/arXiv.2606.06303 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 199 cuvinte
Subiecte conexe
Plug-and-Play Guidance for Discrete Diffusion Models via Gradient-Informed Logit Correction
arXiv · 6 iun. 2026
The Continual Constitutional Convention: How language models revise the documents that govern them
AI Alignment Forum · 13 oct. 2026
Grab a Coffee: Future-Aware Guidance for Discrete Diffusion with Compiled Objectives
arXiv · 30 sept. 2026
Getting Your Guidance Weights Right in diffusion and flow-matching posterior sampling
arXiv · 5 oct. 2026