analiza aiqbench
Modus introduces a novel decoder-only architecture capable of symmetrically processing and generating any combination of modalities within a single unified network. This approach eliminates the need for modality-specific heads or task pipelines while leveraging strong pre-trained priors, offering competitive performance against specialist baselines in cross-modal generation and self-verification tasks. The work advances multimodal modeling by treating all data types equally, enabling flexible chained generation workflows without architectural fragmentation.
MODUS: Decoder-Only Any-to-Any Modeling of Diverse Modalities
arXiv:2607.25948v1 Announce Type: cross Abstract: Any-to-any models predict any modality from any combination of others within a single network, a formulation used in multimodal vision and vision-language models, and increasingly in scientific domains such as ecology and astronomy. Existing any-to-any models are typically trained from scratch using encoder-decoder or diffusion architectures, impacting their performance and preventing them from using strong pre-trained decoder-only models as a prior. In this work, we investigate decoder-only any-to-any multimodal modeling, which treats all modalities symmetrically and supports arbitrary modalities as inputs and outputs without modality-specific heads, losses, or task pipelines. Because every modality is both an input and an output of the same model, the resulting model, named Modus, can support a range of applications, such as chained generation through intermediate modalities or cross-modal self-verification by scoring the model's own outputs with another generated modality. Modus demonstrates strong out-of-the-box performance and is competitive with specialist and multitask baselines using a single model across various benchmarks. All materials are open-sourced at https://modus-multimodal.epfl.ch/.
Abstract:Any-to-any models predict any modality from any combination of others within a single network, a formulation used in multimodal vision and vision-language models, and increasingly in scientific domains such as ecology and astronomy. Existing any-to-any models are typically trained from scratch using encoder-decoder or diffusion architectures, impacting their performance and preventing them from using strong pre-trained decoder-only models as a prior. In this work, we investigate decoder-only any-to-any multimodal modeling, which treats all modalities symmetrically and supports arbitrary modalities as inputs and outputs without modality-specific heads, losses, or task pipelines. Because every modality is both an input and an output of the same model, the resulting model, named Modus, can support a range of applications, such as chained generation through intermediate modalities or cross-modal self-verification by scoring the model's own outputs with another generated modality. Modus demonstrates strong out-of-the-box performance and is competitive with specialist and multitask baselines using a single model across various benchmarks. All materials are open-sourced at this https URL.
Accepted at ICML 2026. Project page: this https URL
Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
arXiv:2607.25948 [cs.CV]
(or arXiv:2607.25948v1 [cs.CV] for this version)
https://doi.org/10.48550/arXiv.2607.25948 arXiv-issued DOI via DataCite (pending registration)
Text integral sub licența deschisă a publicației · 199 cuvinte
Subiecte conexe
Mixture of Decoders for Diverse Dialog Response Generation
arXiv · 2 oct. 2026
[AINews] The Future of Latent Space
Latent Space · 25 sept. 2026
[AINews] Opus 5.5 is good at explainer videos
Latent Space · 29 sept. 2026
Beyond Static Cues: Unified Dynamic Detection of MLLM Backdoors via Dual-Flow Trajectory Analysis
arXiv · 15 oct. 2026