aiqbench analysis
LongMoE introduces a unified framework that simultaneously handles modality missingness and longitudinal dynamics in clinical settings using trajectory-aware Mixture-of-Experts routing. By integrating frequency-domain temporal pattern capture with context-conditioned sparse expert selection, the method improves robustness across datasets like ADNI and MIMIC-IV without sacrificing performance in full-modality scenarios. This advancement addresses critical gaps in existing multimodal models that typically treat visits as static snapshots or assume complete data availability.
LongMoE: Longitudinal Multimodal Learning via Trajectory-Aware Mixture-of-Experts
arXiv:2606.09907v2 Announce Type: replace-cross Abstract: Multimodal clinical learning is increasingly important for integrating diverse patient data, including imaging, text, and personalised health records. However, it faces two fundamental challenges: i) modality missingness, where arbitrary subsets of modalities are unavailable at a given patient visit, ii) longitudinal dynamics, where the diagnostic significance of an observation depends on the patient's evolving disease trajectory over time. Existing methods address these challenges in isolation: missing-modality frameworks treat each visit as an independent static snapshot and discard temporal context, while longitudinal models often assume complete modality availability and degrade under systematic modality incompleteness. We propose LongMoE (Longitudinal Mixture-of-Experts), the unified framework to jointly address both challenges. LongMoE combines a context-aware imputation module with an attentional tokenization module that captures frequency-domain temporal patterns across irregular visit sequences, a trajectory-aware encoder for modeling disease progression, and context-conditioned Sparse MoE routing for patient-specific expert selection. Experiments on ADNI, OASIS-3, and MIMIC-IV show that LongMoE improves robustness under missing or weak contemporaneous modalities and remains competitive in full-modality settings, establishing a strong foundation for longitudinally-aware multimodal clinical learning.
Abstract:Multimodal clinical learning is increasingly important for integrating diverse patient data, including imaging, text, and personalised health records. However, it faces two fundamental challenges: i) modality missingness, where arbitrary subsets of modalities are unavailable at a given patient visit, ii) longitudinal dynamics, where the diagnostic significance of an observation depends on the patient's evolving disease trajectory over time. Existing methods address these challenges in isolation: missing-modality frameworks treat each visit as an independent static snapshot and discard temporal context, while longitudinal models often assume complete modality availability and degrade under systematic modality incompleteness. We propose LongMoE (Longitudinal Mixture-of-Experts), the unified framework to jointly address both challenges. LongMoE combines a context-aware imputation module with an attentional tokenization module that captures frequency-domain temporal patterns across irregular visit sequences, a trajectory-aware encoder for modeling disease progression, and context-conditioned Sparse MoE routing for patient-specific expert selection. Experiments on ADNI, OASIS-3, and MIMIC-IV show that LongMoE improves robustness under missing or weak contemporaneous modalities and remains competitive in full-modality settings, establishing a strong foundation for longitudinally-aware multimodal clinical learning.
Machine Learning (cs.LG); Artificial Intelligence (cs.AI)
arXiv:2606.09907 [cs.LG]
(or arXiv:2606.09907v2 [cs.LG] for this version)
https://doi.org/10.48550/arXiv.2606.09907 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 269 words
Related coverage
LongMoE: Longitudinal Multimodal Learning via Trajectory-Aware Mixture-of-Experts
arXiv · Jun 10, 2026
Risk-Aware Decision-Making for Autonomous Overtaking: A World Model-Based Mixture-of-Experts Framework
arXiv · Sep 2, 2026
Capturing token IDs during agentic interactions for better reinforcement learning
Amazon Science · Jul 9, 2026
Complete Guide to Thinking Machines Inkling
Analytics Vidhya · Jul 20, 2026