Gradient-Momentum Coupling: A Parameter-Space Proxy for Learning Progress

Abstract:Measuring learning progress is at the core of curiosity-driven exploration, which rewards an agent for going where its model is still learning. However, the abstract notion of learning progress is not directly measurable, and existing methods often derive it from the prediction error in the output space. This paper proposes Gradient-Momentum Coupling (GMC), which measures how strongly a sample drives change in the parameter space, given by the normalized absolute product of its gradient with the momentum of previous gradients. Directions of change that persist across samples accumulate in momentum, while noise cancels out. In controlled experiments GMC allocates near uniform priority across tasks with varying levels of noise, where prediction error chases the noisiest, and orders learnable tasks by improvement speed rather than difficulty. On four MiniGrid MultiRoom tasks, substituting GMC for prediction error inside the Intrinsic Curiosity Module (ICM) recovers exploration ICM loses to unpredictable observations.

Machine Learning (cs.LG)

arXiv:2605.05856 [cs.LG]

(or arXiv:2605.05856v2 [cs.LG] for this version)

https://doi.org/10.48550/arXiv.2605.05856 Focus to learn more arXiv-issued DOI via DataCite

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.