Divergence-Based Similarity Function for Multi-View Contrastive Learning

arXiv:2507.06560v5 Announce Type: replace-cross Abstract: Recent success in contrastive learning has sparked growing interest in more effectively leveraging multiple augmented views of data. While prior methods incorporate multiple views at the loss or feature level, they primarily capture pairwise relationships and fail to model the joint structure across all views. In this work, we propose a divergence-based similarity function (DSF) that explicitly captures the joint structure by representing each set of augmented views as a distribution and measuring similarity as the divergence between distributions. Extensive experiments demonstrate that DSF consistently improves performance across diverse tasks, including kNN classification, linear evaluation, transfer learning, and distribution shift, while also achieving greater efficiency than other multi-view methods. Furthermore, we establish a connection between DSF and cosine similarity, and demonstrate that, unlike cosine similarity, DSF operates effectively without the need for tuning a temperature hyperparameter.

Abstract:Recent success in contrastive learning has sparked growing interest in more effectively leveraging multiple augmented views of data. While prior methods incorporate multiple views at the loss or feature level, they primarily capture pairwise relationships and fail to model the joint structure across all views. In this work, we propose a divergence-based similarity function (DSF) that explicitly captures the joint structure by representing each set of augmented views as a distribution and measuring similarity as the divergence between distributions. Extensive experiments demonstrate that DSF consistently improves performance across diverse tasks, including kNN classification, linear evaluation, transfer learning, and distribution shift, while also achieving greater efficiency than other multi-view methods. Furthermore, we establish a connection between DSF and cosine similarity, and demonstrate that, unlike cosine similarity, DSF operates effectively without the need for tuning a temperature hyperparameter.

9 pages, 5 figures. Code and Pretrained Model: this https URL. Published in the proceedings of PAKDD 2026

Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG)

I.2.6; I.4.8; I.5.1

arXiv:2507.06560 [cs.CV]

(or arXiv:2507.06560v5 [cs.CV] for this version)

https://doi.org/10.48550/arXiv.2507.06560 Focus to learn more arXiv-issued DOI via DataCite

Advances in Knowledge Discovery and Data Mining (PAKDD 2026), LNAI 16598, pp. 54-66, Springer, 2026

https://doi.org/10.1007/978-981-92-1462-4_5 Focus to learn more DOI(s) linking to related resources

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.

analiza aiqbench

Divergence-Based Similarity Function for Multi-View Contrastive Learning

Subiecte conexe