analiza aiqbench
The paper tackles a practical problem in training robot policies: multi-step denoising in flow-based vision-language-action models makes policy-gradient RL hard, so the authors propose supervising individual SDE transitions instead of only the final action. Reported gains on LIBERO (for example, 57.6% to 92.0% average success for π₀) are substantial, but they come from simulation benchmarks, start from few-shot fine-tuned models, and have not been independently reproduced. Readers working on embodied AI should watch whether the step-wise supervision generalizes beyond these setups.
$\pi$-StepNFT: Wider Space Needs Finer Steps in Online RL for Flow-based VLAs
arXiv:2603.02083v3 Announce Type: replace Abstract: Flow-based vision-language-action (VLA) models have emerged as a powerful paradigm for embodied control, but multi-step denoising makes policy-gradient online reinforcement learning (RL) difficult because the final-action likelihood induced by the sampler is not directly tractable. We identify an exploration--supervision mismatch within the denoising trajectory, where stochastic differential equation (SDE) sampling explores a wider local stochastic transition space while supervision derived from the final denoised output does not directly use the adjacent transitions observed during sampling. We propose $\pi$-StepNFT (Step-wise Negative-aware Fine-Tuning), a flow-native online RL framework that couples wider stochastic exploration with finer step-wise supervision without computing the final-action likelihood. $\pi$-StepNFT treats observed SDE transitions as supervision units, using variance-normalized transition errors to rank mirrored positive and negative velocity branches and map rollout feedback to local velocity updates. On LIBERO, starting from shared few-shot SFT initializations, $\pi$-StepNFT raises average success from 57.6\% to 92.0\% for $\pi_0$ and from 77.1\% to 94.0\% for $\pi_{0.5}$. On ManiSkill, it achieves the highest reported mean success among the evaluated RL baselines on five test-only visual perturbations. Our project homepage is available at https://wangst0181.github.io/pi-StepNFT/.
Abstract:Flow-based vision-language-action (VLA) models have emerged as a powerful paradigm for embodied control, but multi-step denoising makes policy-gradient online reinforcement learning (RL) difficult because the final-action likelihood induced by the sampler is not directly tractable. We identify an exploration--supervision mismatch within the denoising trajectory, where stochastic differential equation (SDE) sampling explores a wider local stochastic transition space while supervision derived from the final denoised output does not directly use the adjacent transitions observed during sampling. We propose $\pi$-StepNFT (Step-wise Negative-aware Fine-Tuning), a flow-native online RL framework that couples wider stochastic exploration with finer step-wise supervision without computing the final-action likelihood. $\pi$-StepNFT treats observed SDE transitions as supervision units, using variance-normalized transition errors to rank mirrored positive and negative velocity branches and map rollout feedback to local velocity updates. On LIBERO, starting from shared few-shot SFT initializations, $\pi$-StepNFT raises average success from 57.6\% to 92.0\% for $\pi_0$ and from 77.1\% to 94.0\% for $\pi_{0.5}$. On ManiSkill, it achieves the highest reported mean success among the evaluated RL baselines on five test-only visual perturbations. Our project homepage is available at this https URL.
Robotics (cs.RO); Computer Vision and Pattern Recognition (cs.CV)
arXiv:2603.02083 [cs.RO]
(or arXiv:2603.02083v3 [cs.RO] for this version)
https://doi.org/10.48550/arXiv.2603.02083 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 278 cuvinte