Sari la conținut
aiqbenchInformații despre AI & calcul cuantic
ȘtiriAnalizeBenchmark-uriSkill-uriÎntreabă AIÎnvațăDespre
...Se încarcă sesiunea
ȘtiriAnalizeBenchmark-uriSkill-uriÎntreabă AIÎnvațăDespre

Română · beta. Traducerea a fost făcută automat și este încă în curs de revizuire. Citește pagina în engleză

Înapoi la listă
perspectiva aiqbenchMachine learning

analiza aiqbench

This paper introduces MIIB, a novel framework for multimodal self-supervised learning that addresses information heterogeneity by explicitly encoding redundant, unique, and synergistic interactions between data sources like images and text. By utilizing Partial Information Decomposition and entropy-based objectives, the method achieves superior representations without relying on explicit mutual information estimation, outperforming current state-of-the-art contrastive approaches.
1
Sursă care acoperăarXiv

Multimodal Information Interactions Bottleneck for Multimodal Self-Supervised Learning

arXiv·6 noiembrie 2026·2 min de citit·Articolul și perspectiva sunt în engleză.

De la arXiv

Text integral

arXiv:2611.07718v1 Announce Type: cross Abstract: Multimodal self-supervised learning is challenged by heterogeneity, unequal informativeness, and complex information interactions across multimodal data sources such as images, text, audio, and tabular data. While contrastive methods effectively align shared semantics, they typically rely on strong multi-view redundancy assumptions, overlooking modality-specific and synergistic information, thus leading to suboptimal representations. We propose the Multimodal Information Interactions Bottleneck (MIIB), a principled framework grounded in Partial Information Decomposition that explicitly encodes redundant, unique, and synergistic information within multimodal interactions.

MIIB employs a dual-path model architecture for modality-specific encoding and multimodal fusion, optimized using surrogate tractable entropy-based objectives that enforce minimal sufficient representations without requiring explicit mutual information estimation. By leveraging variational bounds and score-based entropy estimation, MIIB efficiently captures complex cross-modal interactions while suppressing superfluous information. Experiments across diverse benchmarks demonstrate consistent improvements over state-of-the-art multimodal self-supervised methods. Code is available at https://github.com/wanglingzaishi/MIIB.

Abstract:Multimodal self-supervised learning is challenged by heterogeneity, unequal informativeness, and complex information interactions across multimodal data sources such as images, text, audio, and tabular data. While contrastive methods effectively align shared semantics, they typically rely on strong multi-view redundancy assumptions, overlooking modality-specific and synergistic information, thus leading to suboptimal representations. We propose the Multimodal Information Interactions Bottleneck (MIIB), a principled framework grounded in Partial Information Decomposition that explicitly encodes redundant, unique, and synergistic information within multimodal interactions.

MIIB employs a dual-path model architecture for modality-specific encoding and multimodal fusion, optimized using surrogate tractable entropy-based objectives that enforce minimal sufficient representations without requiring explicit mutual information estimation. By leveraging variational bounds and score-based entropy estimation, MIIB efficiently captures complex cross-modal interactions while suppressing superfluous information. Experiments across diverse benchmarks demonstrate consistent improvements over state-of-the-art multimodal self-supervised methods. Code is available at this https URL.

15 pages, 3-page supplementary material provided as an ancillary file. Accepted for publication in IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI)

Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG)

arXiv:2611.07718 [cs.CV]

(or arXiv:2611.07718v1 [cs.CV] for this version)

https://doi.org/10.48550/arXiv.2611.07718 arXiv-issued DOI via DataCite (pending registration)

From: Liangjian Wen PhD. [view email] [v1] Thu, 5 Nov 2026 09:37:00 UTC (8,791 KB)

Text integral sub licența deschisă a publicației · 205 cuvinte

Citește pe arXiv ↗Întreabă despre acest articol

Article info

Source

arXiv

Published

6 noiembrie 2026

Read time

2 min

Category

Machine learning
aiqbench
Informații despre AI & calcul cuantic© aiqbench. Toate drepturile rezervate.

Fluxuri:RSS·JSON Feed

LimbăEnglish·Românăbeta
ȘtiriAI & tehnologieCuanticAnalize
ExploreazăBenchmark-uriÎnvață
CompanieDespreÎncredereInformații legaleTermeni și condițiiPolitica de confidențialitateNotificări privind drepturile de autorȘtergerea datelorRaportează conținut

Perspectiva aiqbench, analizele, rezumatul zilnic și răspunsurile din Întreabă sunt scrise de modele AI din sursele afișate, fără revizuire editorială înainte de publicare. Pot conține erori — verifică sursa. Cum este folosit modelul →

  • Știri
  • Analize
  • Întreabă
  • Topuri
  • Cont