analiza aiqbench
LongMoE addresses critical gaps in multimodal clinical AI by jointly handling missing data and temporal dynamics through a unified framework that integrates imputation with trajectory-aware encoding. By leveraging frequency-domain temporal patterns and sparse Mixture-of-Experts routing, the model maintains robustness across irregular visit sequences while remaining competitive in complete data settings. This approach offers a practical foundation for building more resilient systems in healthcare applications where data availability is often inconsistent.
LongMoE: Longitudinal Multimodal Learning via Trajectory-Aware Mixture-of-Experts
arXiv:2606.09907v1 Announce Type: new Abstract: Multimodal clinical learning is increasingly important for integrating diverse patient data, including imaging, text, and personalised health records. However, it faces two fundamental challenges: i) modality missingness, where arbitrary subsets of modalities are unavailable at a given patient visit, ii) longitudinal dynamics, where the diagnostic significance of an observation depends on the patient's evolving disease trajectory over time. Existing methods address these challenges in isolation: missing-modality frameworks treat each visit as an independent static snapshot and discard temporal context, while longitudinal models often assume complete modality availability and degrade under systematic modality incompleteness. We propose LongMoE (Longitudinal Mixture-of-Experts), the unified framework to jointly address both challenges. LongMoE combines a context-aware imputation module with an attentional tokenization module that captures frequency-domain temporal patterns across irregular visit sequences, a trajectory-aware encoder for modeling disease progression, and context-conditioned Sparse MoE routing for patient-specific expert selection. Experiments on ADNI, OASIS-3, and MIMIC-IV show that LongMoE improves robustness under missing or weak contemporaneous modalities and remains competitive in full-modality settings, establishing a strong foundation for longitudinally-aware multimodal clinical learning.
Abstract:Multimodal clinical learning is increasingly important for integrating diverse patient data, including imaging, text, and personalised health records. However, it faces two fundamental challenges: i) modality missingness, where arbitrary subsets of modalities are unavailable at a given patient visit, ii) longitudinal dynamics, where the diagnostic significance of an observation depends on the patient's evolving disease trajectory over time. Existing methods address these challenges in isolation: missing-modality frameworks treat each visit as an independent static snapshot and discard temporal context, while longitudinal models often assume complete modality availability and degrade under systematic modality incompleteness. We propose LongMoE (Longitudinal Mixture-of-Experts), the unified framework to jointly address both challenges. LongMoE combines a context-aware imputation module with an attentional tokenization module that captures frequency-domain temporal patterns across irregular visit sequences, a trajectory-aware encoder for modeling disease progression, and context-conditioned Sparse MoE routing for patient-specific expert selection. Experiments on ADNI, OASIS-3, and MIMIC-IV show that LongMoE improves robustness under missing or weak contemporaneous modalities and remains competitive in full-modality settings, establishing a strong foundation for longitudinally-aware multimodal clinical learning.
Machine Learning (cs.LG); Artificial Intelligence (cs.AI)
arXiv:2606.09907 [cs.LG]
(or arXiv:2606.09907v1 [cs.LG] for this version)
https://doi.org/10.48550/arXiv.2606.09907 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 269 cuvinte
Subiecte conexe
LongMoE: Longitudinal Multimodal Learning via Trajectory-Aware Mixture-of-Experts
arXiv · 1 oct. 2026
Risk-Aware Decision-Making for Autonomous Overtaking: A World Model-Based Mixture-of-Experts Framework
arXiv · 2 sept. 2026
Capturing token IDs during agentic interactions for better reinforcement learning
Amazon Science · 9 iul. 2026
Complete Guide to Thinking Machines Inkling
Analytics Vidhya · 20 iul. 2026