aiqbench analysis
Q-Steer addresses the myopic nature of molecular policy optimization by introducing an action-value steering primitive that guides language models with delayed feedback. By integrating a frozen prefix-action value scorer into sampling logits, this method improves valid-unique scores across multiple backbones without increasing online oracle budgets or altering standard optimizer rules. This approach represents a significant efficiency gain for generating high-quality molecules using existing large-scale molecular foundation models.
Q-Steer: Action-Value Guidance for Molecular Policy Optimization
arXiv:2607.26391v1 Announce Type: new Abstract: Oracle-limited molecular optimization gives reward only after a complete molecule is generated, while each rollout requires many local next-token decisions. This delayed-feedback interface makes molecular policy optimization myopic: an optimizer can learn that a molecule was good without knowing which intermediate actions made it good. We introduce Q-Steer, a rollout-time action-value steering primitive for molecular language models. Q-Steer uses an offline-trained and frozen prefix-action value scorer, PAVS-Q, that estimates the downstream reward of taking a candidate next token under a partial SMILES prefix, then adds a normalized value bonus to sampling logits. The optimizer update rule and online oracle budget are unchanged; the claim is fixed-online-oracle performance, not equal total compute. On PMO23 with a fixed 10,000-call online budget, complete factorial studies across two molecular language-model backbones and four optimizers show that Q-Steer improves mean valid-unique score in all eight backbone-optimizer cells, with positive macro mean-score gains between +0.033 and +0.049 and 18-20 task wins per cell. Mechanism controls show that action identity matters: prefix-broadcast values are nearly neutral, while shuffled action values harm performance. These results support Q-Steer as a reusable rollout-time action-value wrapper that improves average molecular optimization reward across optimizer families and policy backbones without changing the online oracle budget.
Abstract:Oracle-limited molecular optimization gives reward only after a complete molecule is generated, while each rollout requires many local next-token decisions. This delayed-feedback interface makes molecular policy optimization myopic: an optimizer can learn that a molecule was good without knowing which intermediate actions made it good. We introduce Q-Steer, a rollout-time action-value steering primitive for molecular language models. Q-Steer uses an offline-trained and frozen prefix-action value scorer, PAVS-Q, that estimates the downstream reward of taking a candidate next token under a partial SMILES prefix, then adds a normalized value bonus to sampling logits. The optimizer update rule and online oracle budget are unchanged; the claim is fixed-online-oracle performance, not equal total compute. On PMO23 with a fixed 10,000-call online budget, complete factorial studies across two molecular language-model backbones and four optimizers show that Q-Steer improves mean valid-unique score in all eight backbone-optimizer cells, with positive macro mean-score gains between +0.033 and +0.049 and 18-20 task wins per cell. Mechanism controls show that action identity matters: prefix-broadcast values are nearly neutral, while shuffled action values harm performance. These results support Q-Steer as a reusable rollout-time action-value wrapper that improves average molecular optimization reward across optimizer families and policy backbones without changing the online oracle budget.
20 pages, 2 figures, and 25 tables. Includes supplementary experimental results
Machine Learning (cs.LG); Biomolecules (q-bio.BM)
arXiv:2607.26391 [cs.LG]
(or arXiv:2607.26391v1 [cs.LG] for this version)
https://doi.org/10.48550/arXiv.2607.26391 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 310 words
Related coverage
Adjoint Guidance Flow: Amortized Critic Guidance for VLA Policies
arXiv · Sep 30, 2026
Safe Real-Time Policy Steering via Noise-Space Trajectory Optimization for One-Step Generative Policies
arXiv · Sep 21, 2026
PredActor: Predictive Action Diffusion for Steerable Onboard Humanoid Control
arXiv · Sep 22, 2026
TTT-VLA: Test-Time Latent Prompt Optimization for Vision-Language-Action Models
arXiv · Jun 3, 2026