analiza aiqbench
GOMA addresses a critical bottleneck in multimodal retrieval by decoupling identity preservation from relational aggregation, effectively solving the problem where standard graph regularization blurs distinct object features. By employing dual embeddings with restart propagation, the method maintains sharp content distinctions while leveraging cross-object relationships, achieving state-of-the-art results across diverse datasets and tasks. This structural approach offers a robust framework for improving retrieval accuracy in complex multimodal environments without sacrificing specificity.
GOMA: Toward Structure-Driven Multimodal Alignment from a Graph Signal Smoothing Perspective
arXiv:2605.15723v2 Announce Type: replace Abstract: Multimodal retrieval uses images, text, and object relationships to answer different questions about the same collection. A query may seek an object's paired description, another object in the same category, or an object connected by an observed relationship. These goals rely on different notions of relevance. Paired matching requires object-specific distinctions, whereas cross-object retrieval benefits from relational agreement. Existing methods learn strong cross-modal correspondence or propagate information over a graph, but a shared output regularized toward neighbors can weaken identity distinctions. Moreover, gains from graph regularization can diminish after uniform graph propagation. We introduce Graph-Optimized Multimodal Alignment (GOMA), which assigns these roles to two connected embeddings. Each modality produces a content embedding directly supervised for paired identity and a semantic embedding jointly trained with cross-modal pairs and observed relationships. For complete records, both embeddings form an initial fused representation, semantic agreement sets positive weights on observed edges, and restart graph propagation reinjects this initial signal. This design lets a jointly trained model support single-modality and dual-attribute retrieval through a task-specific readout. Across six datasets and four tasks, GOMA achieves state-of-the-art performance on all 14 primary measures against 14 external methods. Controlled comparisons further show how separate supervision, graph regularization, and semantic-guided graph propagation shape the final representation and align the learned signal with each retrieval target.
Abstract:Multimodal retrieval uses images, text, and object relationships to answer different questions about the same collection. A query may seek an object's paired description, another object in the same category, or an object connected by an observed relationship. These goals rely on different notions of relevance. Paired matching requires object-specific distinctions, whereas cross-object retrieval benefits from relational agreement. Existing methods learn strong cross-modal correspondence or propagate information over a graph, but a shared output regularized toward neighbors can weaken identity distinctions. Moreover, gains from graph regularization can diminish after uniform graph propagation. We introduce Graph-Optimized Multimodal Alignment (GOMA), which assigns these roles to two connected embeddings. Each modality produces a content embedding directly supervised for paired identity and a semantic embedding jointly trained with cross-modal pairs and observed relationships. For complete records, both embeddings form an initial fused representation, semantic agreement sets positive weights on observed edges, and restart graph propagation reinjects this initial signal. This design lets a jointly trained model support single-modality and dual-attribute retrieval through a task-specific readout. Across six datasets and four tasks, GOMA achieves state-of-the-art performance on all 14 primary measures against 14 external methods. Controlled comparisons further show how separate supervision, graph regularization, and semantic-guided graph propagation shape the final representation and align the learned signal with each retrieval target.
Machine Learning (cs.LG); Computer Vision and Pattern Recognition (cs.CV)
arXiv:2605.15723 [cs.LG]
(or arXiv:2605.15723v2 [cs.LG] for this version)
https://doi.org/10.48550/arXiv.2605.15723 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 314 cuvinte
Subiecte conexe
PRISM: A Category-Theoretic Framework for Measuring and Refining Multimodal Analogies
arXiv · 2 oct. 2026
News
Quantum Computing Report · 25 iul. 2026
Video Understanding by Design: How Datasets Shape Video Models
arXiv · 9 iun. 2026
Toward Federated Multimodal Graph Foundation Models: A Topology-Aware Multimodal Alignment Framework
arXiv · 20 iul. 2026