analiza aiqbench
This paper introduces FIERCE, a reinforcement learning framework that accelerates the refinement of specialized robot policies by leveraging a unified progress-failure evaluator initialized with generalist models. By eliminating the need for dense manual rewards or continuous simulator access during training, the method significantly reduces data costs and deployment complexity for contact-rich physical tasks. The approach represents a meaningful advancement in efficient robot learning architectures, directly addressing bottlenecks in scaling embodied AI systems.
FIERCE: From Generalist Robot Policies to Fast Specialists via Progress-Failure Feedback
arXiv:2609.18651v1 Announce Type: new Abstract: Generalist robot policies offer useful initialization, but refining compact specialists through limited physical interaction requires informative learning feedback. We present FIERCE, a generalist-initialized reinforcement learning framework centered on a unified, task-adaptive progress-failure evaluator. Its architecture shares an observation-language representation between an observed-progress head and an action-conditioned latent predictor whose past and current predictions feed a causal sequence head for task-failure estimation. Joint supervision from progress and preference labels, synchronized commands and observations, and terminal outcomes trains the evaluator; target-task rollouts support adaptation and calibration. Fixed evaluator snapshots provide progress shaping and failure-risk penalties alongside independently verified terminal rewards, while evaluator and policy updates alternate as new experience is collected. Refinement requires neither continued generalist action queries nor a dedicated target-task simulator or manually annotated dense rewards. Only the compact specialist is retained at deployment. The evaluation separates feedback quality, policy-learning efficiency, and deployment cost across simulation and two contact-rich real tasks. Code, model weights, and data-restoration tools are released at https://github.com/ar-mine/FIERCE.
Abstract:Generalist robot policies offer useful initialization, but refining compact specialists through limited physical interaction requires informative learning feedback. We present FIERCE, a generalist-initialized reinforcement learning framework centered on a unified, task-adaptive progress-failure evaluator. Its architecture shares an observation-language representation between an observed-progress head and an action-conditioned latent predictor whose past and current predictions feed a causal sequence head for task-failure estimation. Joint supervision from progress and preference labels, synchronized commands and observations, and terminal outcomes trains the evaluator; target-task rollouts support adaptation and calibration. Fixed evaluator snapshots provide progress shaping and failure-risk penalties alongside independently verified terminal rewards, while evaluator and policy updates alternate as new experience is collected. Refinement requires neither continued generalist action queries nor a dedicated target-task simulator or manually annotated dense rewards. Only the compact specialist is retained at deployment. The evaluation separates feedback quality, policy-learning efficiency, and deployment cost across simulation and two contact-rich real tasks. Code, model weights, and data-restoration tools are released at this https URL.
Robotics (cs.RO)
arXiv:2609.18651 [cs.RO]
(or arXiv:2609.18651v1 [cs.RO] for this version)
https://doi.org/10.48550/arXiv.2609.18651 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 256 cuvinte
Subiecte conexe
The Machines that Make the Machines
NVIDIA · 7 oct. 2026
Runway’s WorldPrompt and the Engineering of Real-Time Worlds
Latent Space · 25 sept. 2026
How to Evaluate General-Purpose Robot Policies for Real-World Deployment
NVIDIA · 12 iul. 2026
Extreme-RGMT: Continual Learning of Highly Dynamic Skills for Robust Generalist Humanoid Control
arXiv · 23 iul. 2026