analiza aiqbench
WASP-Mem addresses a critical bottleneck in LLM agent deployment by converting execution traces and human diagnoses into persistent, wiki-based procedural memory without requiring retraining. The study demonstrates that this approach significantly improves checkpoint pass rates on enterprise benchmarks while reducing storage needs, proving that human effort should focus on diagnosing failures rather than defining rigid schemas. This work advances the field of continual learning for agents, offering a practical framework to handle undocumented behaviors and state-dependent recovery paths in complex workflows.
WASP-Mem: Let Agent Organize Procedural Memory. Human Spend Effort on Diagnosis
arXiv:2611.04003v1 Announce Type: cross Abstract: LLM agents struggle to automate complex enterprise procedures because static, pre-built harness cannot capture proprietary policies, undocumented behaviors, or state-dependent recovery paths that surface only during execution. Procedural memory accrued from agent experience offers a training-free path to closing this gap -- yet unlike episodic records or semantic facts, it must consolidate repeated successes and diagnosed failures into reusable actions and recovery strategies. We introduce WASP-Mem (Wiki-based Agent-Synthesized Procedural Memory), a nimble but effective method that distills execution traces and feedback into persistent, interlinked wiki pages, enabling continual agent learning. On TheAgentCompany benchmark, WASP-Mem raises the checkpoint pass rate by 4.6-6.1% over the strongest baseline we compared across three models while requiring 57-61% less storage. We further present a systematic controlled study spanning consolidation method, feedback type, trace ingestion scope, and refinement rounds. Our results reveal that prescriptive knowledge organization such as human defined schemas and explicit deduplication rules hurts performance rather than closing the procedural-knowledge gap. In contrast, human diagnosis converts failed traces into corrective knowledge and drives sustained improvement: a single round of human diagnosis outperforms four rounds of self-judgment by 7.0-10.9%. These findings point to a simple division of labor: let agents organize procedural memory, and invest human effort where it matters most -- diagnosis.
Title:WASP-Mem: Let Agent Organize Procedural Memory. Human Spend Effort on Diagnosis
Abstract:LLM agents struggle to automate complex enterprise procedures because static, pre-built harness cannot capture proprietary policies, undocumented behaviors, or state-dependent recovery paths that surface only during execution. Procedural memory accrued from agent experience offers a training-free path to closing this gap -- yet unlike episodic records or semantic facts, it must consolidate repeated successes and diagnosed failures into reusable actions and recovery strategies. We introduce WASP-Mem (Wiki-based Agent-Synthesized Procedural Memory), a nimble but effective method that distills execution traces and feedback into persistent, interlinked wiki pages, enabling continual agent learning. On TheAgentCompany benchmark, WASP-Mem raises the checkpoint pass rate by 4.6-6.1% over the strongest baseline we compared across three models while requiring 57-61% less storage. We further present a systematic controlled study spanning consolidation method, feedback type, trace ingestion scope, and refinement rounds. Our results reveal that prescriptive knowledge organization such as human defined schemas and explicit deduplication rules hurts performance rather than closing the procedural-knowledge gap. In contrast, human diagnosis converts failed traces into corrective knowledge and drives sustained improvement: a single round of human diagnosis outperforms four rounds of self-judgment by 7.0-10.9%. These findings point to a simple division of labor: let agents organize procedural memory, and invest human effort where it matters most -- diagnosis.
24 pages (10 main + references and appendix), 6 figures, 14 tables. Peizhi Niu and Ge Shi contributed equally
Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Machine Learning (cs.LG)
arXiv:2611.04003 [cs.AI]
(or arXiv:2611.04003v1 [cs.AI] for this version)
https://doi.org/10.48550/arXiv.2611.04003 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 339 cuvinte
Subiecte conexe
Sensor Diagnosis for Fuel and Emissions Monitoring in Heavy-Duty Diesel Vehicles
arXiv · 16 oct. 2026
SMI: Ontology-Grounded Semantic IDs for Hierarchy-Aware Medical Concept Representation
arXiv · 15 oct. 2026
Multilingual Verifier Bias in RLVR: Benchmark, Rollout Diagnosis, and the Cross-Lingual Selection Bottleneck
arXiv · 24 aug. 2026
Decomposing Error and Style in Automated Clinical Coding
arXiv · 22 sept. 2026