analiza aiqbench
This arXiv preprint studies catastrophic forgetting in unified multimodal models, which handle both understanding and generation across modalities, when they are trained continually on new tasks. Its main claims are that more modality-specific parameter isolation reduces cross-modal interference, that the generation side is more fragile than the understanding side, and that post-training skills such as personalized generation degrade sharply (84% to 90%) while pretraining capabilities hold up. For practitioners, the useful takeaway is that continual-learning plans for multimodal systems should treat post-training adaptations as the most vulnerable part of the model, though these results come from a single preprint and still need independent replication.
Continual Adaptation Across Modalities: Learning and Forgetting in Unified Multimodal Models
arXiv:2610.38436v1 Announce Type: new Abstract: Unified multimodal models (UMMs), which can process and generate diverse modalities within a single framework, have recently emerged as a powerful paradigm. However, when these models undergo continual learning on multiple tasks across different modalities, their learning and forgetting dynamics remain poorly understood. In this paper, we systematically investigate how architectural design and adaptation method shape intra-modal learning and cross-modal interference between the generation and understanding sides. Specifically, we report three findings. First, the severity of cross-modal interference decreases with the degree of modality-specific parameter isolation. Second, a within-model asymmetry: the generation side of a UMM is markedly more fragile than its understanding side. Third, what post-training adds is the first to be forgotten: purely post-training capabilities such as personalized generation collapse by 84\% to 90\%, while pretraining capabilities remain stable. Together, these results clarify how parameter isolation and post-training fragility shape UMMs' learning and forgetting dynamics, offering a starting point for developing continually evolving UMMs.
Abstract:Unified multimodal models (UMMs), which can process and generate diverse modalities within a single framework, have recently emerged as a powerful paradigm. However, when these models undergo continual learning on multiple tasks across different modalities, their learning and forgetting dynamics remain poorly understood. In this paper, we systematically investigate how architectural design and adaptation method shape intra-modal learning and cross-modal interference between the generation and understanding sides. Specifically, we report three findings. First, the severity of cross-modal interference decreases with the degree of modality-specific parameter isolation. Second, a within-model asymmetry: the generation side of a UMM is markedly more fragile than its understanding side. Third, what post-training adds is the first to be forgotten: purely post-training capabilities such as personalized generation collapse by 84\% to 90\%, while pretraining capabilities remain stable. Together, these results clarify how parameter isolation and post-training fragility shape UMMs' learning and forgetting dynamics, offering a starting point for developing continually evolving UMMs.
Machine Learning (cs.LG)
arXiv:2610.38436 [cs.LG]
(or arXiv:2610.38436v1 [cs.LG] for this version)
https://doi.org/10.48550/arXiv.2610.38436 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 250 cuvinte
Subiecte conexe
MAD: Modality-Adaptive Decoding for Mitigating Cross-Modal Hallucinations in Multimodal Large Language Models
arXiv · 14 sept. 2026
MAND: Modality-Aware Novelty Detection for Open-World Egocentric Activity Recognition
arXiv · 16 iun. 2026
FedTaste: Topology-Aware Structural Transfer for Multimodal Federated Learning with Missing Modalities
arXiv · 28 iul. 2026
TTSD-FAR: Test-Time Self-Distillation with Fisher-Anchored Restoration for Missing-Modality Emotion Recognition in LVLMs
arXiv · 21 aug. 2026