analiza aiqbench
This research introduces "probe guidance," a technique that leverages frozen internal states of diffusion models to enhance flow matching without incurring extra inference costs. By ensuring weak and strong models share similar dynamics, the method achieves new state-of-the-art results on unconditional generation and multiple-choice benchmarks for continuous diffusion language models. The study also clarifies the underlying mechanisms of autoguidance, revealing that effective weak checkpoints must originate from low-entropy training regions.
How to Guide Your Language Flow
arXiv:2609.19356v1 Announce Type: cross Abstract: We introduce a new method to guide flow matching models. Our approach, which we call probe guidance, uses the frozen internal states of an existing diffusion model to construct a guidance signal. This works using a similar principle as autoguidance, but eliminates the need for an additional forward pass at inference time and provides a reliable path to ensure that the weak and strong model share similar dynamics. We apply and benchmark this method on continuous diffusion language models, where probe guidance sets a new state-of-the-art performance on unconditional generation. When applied to a 1.7B diffusion language model, probe guidance consistently improves on multiple choice question answering benchmarks. Using our probes, we study the traditional autoguidance setting where the strong model is a weak checkpoint, and find that the weak model must come from a low-entropy region of training. These findings both provide a practical way to improve diffusion language models and shed light on the actual mechanism behind autoguidance, which is currently poorly understood.
Abstract:We introduce a new method to guide flow matching models. Our approach, which we call probe guidance, uses the frozen internal states of an existing diffusion model to construct a guidance signal. This works using a similar principle as autoguidance, but eliminates the need for an additional forward pass at inference time and provides a reliable path to ensure that the weak and strong model share similar dynamics. We apply and benchmark this method on continuous diffusion language models, where probe guidance sets a new state-of-the-art performance on unconditional generation. When applied to a 1.7B diffusion language model, probe guidance consistently improves on multiple choice question answering benchmarks. Using our probes, we study the traditional autoguidance setting where the strong model is a weak checkpoint, and find that the weak model must come from a low-entropy region of training. These findings both provide a practical way to improve diffusion language models and shed light on the actual mechanism behind autoguidance, which is currently poorly understood.
Machine Learning (cs.LG); Artificial Intelligence (cs.AI)
arXiv:2609.19356 [cs.LG]
(or arXiv:2609.19356v1 [cs.LG] for this version)
https://doi.org/10.48550/arXiv.2609.19356 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 262 cuvinte
Subiecte conexe
AERMANI-PLACE: Language Guided Object Placement with Aerial Manipulators
arXiv · 16 iun. 2026
From Sign Language Generation to Humanoid Execution: Vision-Language Guided Retargeting with Collision Mitigation
arXiv · 21 iul. 2026
FRAME: Factored Retrieval via Attribute Readouts for Object-Centric Scene Memory
arXiv · 9 sept. 2026
OPD-IAD: From Language Judgment to Industrial Anomaly Detection via On-Policy Self-Distillation
arXiv · 23 iul. 2026