aiqbench analysis
This study challenges the prevailing heuristic in self-supervised learning by demonstrating that richer spectral features do not consistently improve generalization in molecular property prediction. The research reveals that only specific representations, such as ECFP-based kernels and global 3D descriptors, maintain a positive correlation with performance, while local 3D representations often degrade results. These findings provide crucial guidance for optimizing kernel methods in low-data scientific regimes where traditional model complexity assumptions fail.
Spectral Analysis of Molecular Features: When Richer Features Do Not Guarantee Better Generalization
arXiv:2510.14217v3 Announce Type: replace Abstract: The spectral properties of feature embeddings offer critical insights into model generalization and representation quality. While deep learning models are widely used for molecular property prediction, kernel methods remain competitive in low-data regimes, yet their spectral behavior is largely unexplored. We present the first comprehensive spectral analysis of kernel ridge regression across diverse representations, including molecular fingerprints (ECFP), pretrained transformers, graph neural networks, and 3D descriptors, evaluated on QM9, MoleculeNet, and Therapeutics Data Commons benchmarks. Surprisingly, richer spectral features do not consistently yield better generalization performance, contradicting common representation heuristics used in self-supervised learning (SSL). Across four spectral metrics, only ECFP-based kernels show a strictly positive correlation with performance, particularly SSE, ID, and SR on MoleculeNet. Global 3D representations exhibit mixed behavior; transformer-based representations are consistently positive across all four metrics, whereas local 3D representations show a negative trend and no significant correlations. Truncation analysis further emphasizes this disparity: for local 3D representations on thermodynamic targets, fewer than 2% of eigenvalues, and occasionally as few as 0.02%, are needed to recover 95% of performance, whereas ECFP and transformer kernels require significantly more. By demonstrating a strong dependence on both task and representation, our results challenge the heuristic that richer spectra inherently improve generalization, providing new guidance for evaluating representations in SSL and in label-limited scientific tasks.
Abstract:The spectral properties of feature embeddings offer critical insights into model generalization and representation quality. While deep learning models are widely used for molecular property prediction, kernel methods remain competitive in low-data regimes, yet their spectral behavior is largely unexplored. We present the first comprehensive spectral analysis of kernel ridge regression across diverse representations, including molecular fingerprints (ECFP), pretrained transformers, graph neural networks, and 3D descriptors, evaluated on QM9, MoleculeNet, and Therapeutics Data Commons benchmarks. Surprisingly, richer spectral features do not consistently yield better generalization performance, contradicting common representation heuristics used in self-supervised learning (SSL). Across four spectral metrics, only ECFP-based kernels show a strictly positive correlation with performance, particularly SSE, ID, and SR on MoleculeNet. Global 3D representations exhibit mixed behavior; transformer-based representations are consistently positive across all four metrics, whereas local 3D representations show a negative trend and no significant correlations. Truncation analysis further emphasizes this disparity: for local 3D representations on thermodynamic targets, fewer than 2% of eigenvalues, and occasionally as few as 0.02%, are needed to recover 95% of performance, whereas ECFP and transformer kernels require significantly more. By demonstrating a strong dependence on both task and representation, our results challenge the heuristic that richer spectra inherently improve generalization, providing new guidance for evaluating representations in SSL and in label-limited scientific tasks.
Machine Learning (cs.LG); Chemical Physics (physics.chem-ph)
arXiv:2510.14217 [cs.LG]
(or arXiv:2510.14217v3 [cs.LG] for this version)
https://doi.org/10.48550/arXiv.2510.14217 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 311 words
Related coverage
Spectral Analysis of Molecular Features: When Richer Features Do Not Guarantee Better Generalization
arXiv · Jun 16, 2026
Rank-Efficient LoRA via Joint Tangent-Space Optimization under Isotropic Curvature
arXiv · Sep 14, 2026
Researchers Simulate Molecule Spectra at Any Temperature Rapidly
Quantum Zeitgeist · Sep 2, 2026
Exploding variance of means of exponentials: least-squares to the rescue – Machine Learning Research Blog
francisbach.com · Sep 27, 2026