aiqbench analysis
This 100-page tutorial consolidates decades of work on recovering shared structure from multiple views, spanning information theory, classical statistics, and signal processing. Its most relevant parts for ML readers are the treatment of nonlinear CCA (kernel, deep, and probabilistic variants), the identifiability conditions for the common factor, and the links to self-supervised learning. It is a long theoretical survey rather than a new empirical result, so its value lies in clarifying which methods are exactly equivalent and which are only analogous.
From Common Information to Canonical Correlation Analysis and Beyond: Bridging Information Theory, Signal Processing, and Machine Learning
arXiv:2610.28978v1 Announce Type: cross Abstract: Many problems in signal processing, machine learning, and neuroimaging provide several jointly observed measurements of one process, each mixing a component common to all of them with a component private to itself, and require the common component to be recovered without supervision. This problem has been studied in several communities, largely independently. Statisticians proposed canonical correlation analysis (CCA), introduced by Hotelling in 1936. Information theorists proposed two notions of common information, due to G\'acs and K\"orner and to Wyner. Engineers developed subspace methods for sensor arrays and multiuser communications to design multiview receivers. This tutorial presents the main results and tools each community developed for this problem in a single exposition, and explains the connections that emerge where they meet, distinguishing exact equivalences from special cases and analogies. We develop the information-theoretic foundation from first principles. We then introduce CCA and its extension to several views, followed by the engineering thread from blind multiuser detection to CCA as a blind receiver. Next, we turn to nonlinear CCA, covering kernel, deep, and probabilistic formulations, the conditions under which the common factor is identifiable, and connections to self-supervised learning. We conclude with a discussion of further extensions and open directions. The exposition is self-contained, so that readers from each of these communities can follow the parts developed by the others.
Abstract:Many problems in signal processing, machine learning, and neuroimaging provide several jointly observed measurements of one process, each mixing a component common to all of them with a component private to itself, and require the common component to be recovered without supervision. This problem has been studied in several communities, largely independently. Statisticians proposed canonical correlation analysis (CCA), introduced by Hotelling in 1936. Information theorists proposed two notions of common information, due to Gács and Körner and to Wyner. Engineers developed subspace methods for sensor arrays and multiuser communications to design multiview receivers. This tutorial presents the main results and tools each community developed for this problem in a single exposition, and explains the connections that emerge where they meet, distinguishing exact equivalences from special cases and analogies. We develop the information-theoretic foundation from first principles. We then introduce CCA and its extension to several views, followed by the engineering thread from blind multiuser detection to CCA as a blind receiver. Next, we turn to nonlinear CCA, covering kernel, deep, and probabilistic formulations, the conditions under which the common factor is identifiable, and connections to self-supervised learning. We conclude with a discussion of further extensions and open directions. The exposition is self-contained, so that readers from each of these communities can follow the parts developed by the others.
Tutorial, 100 pages, 11 figures. Accepted for publication in Entropy, Special Issue "Foundations and Frontiers of Information Theory" in honor of H. Vincent Poor's 75th birthday
Information Theory (cs.IT); Machine Learning (cs.LG); Signal Processing (eess.SP); Machine Learning (stat.ML)
arXiv:2610.28978 [cs.IT]
(or arXiv:2610.28978v1 [cs.IT] for this version)
https://doi.org/10.48550/arXiv.2610.28978 arXiv-issued DOI via DataCite (pending registration)
Full text under the publisher's open licence · 271 words
Related coverage
A memory-information window in finite-duration quantum control
arXiv · Sep 21, 2026
Mutual information across the commensurate--incommensurate crossover and near the Kosterlitz--Thouless point of the spin-$\tfrac12$ $J_1$--$J_2$ Heisenberg chain
arXiv · Oct 14, 2026
SEHFS: Structural Entropy-Guided High-Order Correlation Learning for Multi-View Multi-Label Feature Selection
arXiv · Oct 21, 2026
Explainability of Complex AI Models with Correlation Impact Ratio
arXiv · Oct 2, 2026