Weighted Low-Rank Matrix Approximation: Acceleration and Applications

arXiv:2109.11057v2 Announce Type: replace-cross Abstract: Weighted low-rank matrix approximation (WLRMA) generalizes classical low-rank approximation and matrix completion by allowing arbitrary elementwise weights. Such formulations arise naturally in a broad class of statistical models, including generalized linear low-rank models, where WLRMA serves as the computational primitive for parameter estimation. Despite its broad applicability, efficient optimization methods for general WLRMA remain relatively underdeveloped. In this paper, we formulate both the rank-constrained and nuclear-norm WLRMA problems within a unified first-order optimization framework by showing that the corresponding iterative algorithms are projected and proximal gradient methods. Building on this perspective, we develop accelerated algorithms based on Nesterov momentum and Anderson acceleration, together with a regularized Anderson scheme that improves numerical stability for non-convex problems. We further propose scalable implementations for large sparse data matrices and introduce a practical effective-rank criterion that provides a meaningful correspondence between rank-constrained and nuclear-norm solutions. We further show that fitting generalized linear low-rank models can be reduced to a sequence of weighted low-rank matrix approximation problems, allowing the proposed algorithms to be used as computational building blocks for their estimation. Simulation studies demonstrate substantial computational gains achieved by the proposed accelerated algorithms. Applications to the MovieLens dataset further illustrate the proposed framework for matrix completion, heteroscedastic Gaussian low-rank modeling, and logistic low-rank modeling.

Abstract:Weighted low-rank matrix approximation (WLRMA) generalizes classical low-rank approximation and matrix completion by allowing arbitrary elementwise weights. Such formulations arise naturally in a broad class of statistical models, including generalized linear low-rank models, where WLRMA serves as the computational primitive for parameter estimation. Despite its broad applicability, efficient optimization methods for general WLRMA remain relatively underdeveloped. In this paper, we formulate both the rank-constrained and nuclear-norm WLRMA problems within a unified first-order optimization framework by showing that the corresponding iterative algorithms are projected and proximal gradient methods. Building on this perspective, we develop accelerated algorithms based on Nesterov momentum and Anderson acceleration, together with a regularized Anderson scheme that improves numerical stability for non-convex problems. We further propose scalable implementations for large sparse data matrices and introduce a practical effective-rank criterion that provides a meaningful correspondence between rank-constrained and nuclear-norm solutions. We further show that fitting generalized linear low-rank models can be reduced to a sequence of weighted low-rank matrix approximation problems, allowing the proposed algorithms to be used as computational building blocks for their estimation. Simulation studies demonstrate substantial computational gains achieved by the proposed accelerated algorithms. Applications to the MovieLens dataset further illustrate the proposed framework for matrix completion, heteroscedastic Gaussian low-rank modeling, and logistic low-rank modeling.

Machine Learning (stat.ML); Machine Learning (cs.LG); Methodology (stat.ME)

arXiv:2109.11057 [stat.ML]

(or arXiv:2109.11057v2 [stat.ML] for this version)

https://doi.org/10.48550/arXiv.2109.11057 Focus to learn more arXiv-issued DOI via DataCite

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.

analiza aiqbench

Weighted Low-Rank Matrix Approximation: Acceleration and Applications

Subiecte conexe