perspectiva aiqbenchProduse AI

EgoVine introduces a post-training framework that compiles real-world egocentric video data into executable symbolic environments, enabling agents to explore alternative actions and recover from mistakes during long-horizon planning. By jointly training action generation with consequence prediction, the method allows compact planners to learn from failures and sampled continuations, achieving superior performance on complex household procedure benchmarks compared to Flash LLM baselines. This approach effectively bridges static video demonstrations with iterative learning processes that account for partial observation and delayed consequences.

Sursă care acoperăarXiv

Ce este aceastaLectura unui model asupra articolului și a acoperirii conexe, scrisă pentru aiqbench. Nu sunt cuvintele publicației.

EgoVine: Post-Training with Branching Experience Compiled from Real-World Videos

arXiv20 octombrie 20261 min de cititArticolul și perspectiva sunt în engleză.

arXiv:2610.23907v1 Announce Type: cross Abstract: Real-world egocentric videos capture multi-step household procedures, but their recorded trajectories leave alternative actions and recovery from the learner's own mistakes largely unexplored. We introduce EgoVine, a post-training framework that turns a fixed demonstration set into interactive experience for long-horizon planning. EgoVine operates in executable symbolic environments compiled from real-world video annotations, where an agent must coordinate object manipulations under partial observation and account for delayed consequences. Restoring a decision state allows the learner to explore alternative actions and compare their eventual outcomes. Union jointly trains action generation and consequence prediction in one model, using the same executions to supervise candidate comparison and recovery. Verified repairs add recovery targets from failed attempts, while comparisons among sampled continuations guide policy updates. The updated planner then returns to the environments, with its own decisions and mistakes shaping the next round of experience. On a shared benchmark, EgoVine-1.5B achieves 81.8% success on Main planning tasks and 66.7% on longer task compositions, outperforming all evaluated Flash LLM baselines on both. EgoVine connects real demonstrations to an iterative learning process that trains compact planners on choices and futures beyond the recorded trajectories.

Citește articolul complet pe arXiv

Citește pe arXiv ↗Întreabă despre acest articol