aiqbench analysis
This research introduces "Anchor Divergences," a novel framework that leverages information geometry and exponential families to adapt the semantic geometry of vector representations based on context. By moving beyond fixed cosine similarity, the method allows contrastive learning models to specialize their metric spaces for specific tasks like retrieval, offering a more nuanced approach to measuring semantic similarity. The work represents a significant theoretical advancement in how machine learning systems handle context-dependent relationships within high-dimensional embedding spaces.
Anchor Divergence for Semantic Geometry in Contrastive Learning
arXiv:2610.06919v1 Announce Type: cross Abstract: This paper concerns how semantic context determines geometry in learned vector representations. Similarity is typically measured using cosine similarity, which provides a single fixed geometry. Semantic similarity, however, is inherently context dependent: two images may be similar because they depict the same object, share a visual style, or are relevant to the same clinical finding. We show that contrastive representations naturally encompass a family of geometries that can be specialized to particular semantic structure. The key idea is to use an interplay between contrastive learning, exponential families, and information geometry to establish a correspondence between probability distributions over "anchors" and Bregman geometries on the representation space. We use this correspondence to define "Anchor Divergences", a method for specifying context-specific semantic geometries on fixed representations. Under this correspondence, modeling the anchor distribution models the geometry itself. Experiments on retrieval show that anchor divergences provide an effective and efficient way to specify context-specific semantic similarity.
Abstract:This paper concerns how semantic context determines geometry in learned vector representations. Similarity is typically measured using cosine similarity, which provides a single fixed geometry. Semantic similarity, however, is inherently context dependent: two images may be similar because they depict the same object, share a visual style, or are relevant to the same clinical finding. We show that contrastive representations naturally encompass a family of geometries that can be specialized to particular semantic structure. The key idea is to use an interplay between contrastive learning, exponential families, and information geometry to establish a correspondence between probability distributions over "anchors" and Bregman geometries on the representation space. We use this correspondence to define "Anchor Divergences", a method for specifying context-specific semantic geometries on fixed representations. Under this correspondence, modeling the anchor distribution models the geometry itself. Experiments on retrieval show that anchor divergences provide an effective and efficient way to specify context-specific semantic similarity.
Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG); Machine Learning (stat.ML)
arXiv:2610.06919 [cs.AI]
(or arXiv:2610.06919v1 [cs.AI] for this version)
https://doi.org/10.48550/arXiv.2610.06919 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 259 words
Related coverage
Game Design Creativity evals
LessWrong · Oct 12, 2026
A General Framework for Decision Trees via Bregman Divergences
arXiv · Jun 15, 2026
Mirror Descent Beyond Euclidean Stability: An Exponential Separation in Initialization Sensitivity
arXiv · Jun 12, 2026
CAMMAR: Culture-Aware Matryoshka for Metaphorical Arabic Representations
arXiv · Jul 20, 2026