analiza aiqbench
CoM$^3$eT advances medical AI by unifying pathology and radiology tasks within a single foundation model, significantly improving generalization across sparse and dense prediction formats. The architecture achieves near-full-model performance with minimal parameter training (under 2.5%) while enabling effective federated learning on consumer-grade hardware without centralized data pooling. This approach addresses critical barriers in medical imaging by reducing computational overhead and expanding applicability beyond single-specialty constraints.
CoM$^3$eT: A foundation model for medical image analysis through federated, multidimensional context integration
arXiv:2608.16268v1 Announce Type: cross Abstract: Medical foundation models improve generalization when training AI models with limited labeled data, but remain confined to a single specialty, such as pathology or radiology, and to either sparse or dense outputs, such as classification or segmentation. Here, we present CoM$^3$eT (Co-representation Multidimensional Multitask Medical Transformer), a medical vision foundation model that unifies pathology and radiology, sparse and dense predictions, and two- and higher-dimensional inputs by modeling multidimensional context with attention. CoM$^3$eT outperformed other medical foundation models in an open competition spanning five tomographic, four whole-specimen, and three two-dimensional datasets, covering sparse and dense prediction tasks as well as report generation. When adapted across diverse clinical applications, training fewer than 2.5% of parameters achieved performance comparable to full fine-tuning, enabling research without access to high-performance GPU clusters. Applied to federated learning across hospitals, this approach achieved performance comparable to pooled-data training over internet connections and with consumer-grade hardware.
Authors:J. Raphael Schäfer, Kai Geissler, Till Nicke, Chiara Tappermann, Karoline Heber, Eike Petersen, Habib Mergan, Lars Ole Schwen, Nick Weiss, Annika Gerken, Jan Hendrik Moltz, Tom Bisson, Isil Dogan O, Tim-Rasmus Kiehl, Norman Zerbe, Sefer Elezkurtaj, Robin S. Mayer, Nadine Flinner, Peter Wild, Isabel Dahm, Felix Peisen, Heinrich von Busch, Robert Grimm, Sebastian Arndt, Lisa Siegler, Matthias Stefan May, Antje Prasse, Natalia Artysh, Fabian Kiessling, Johannes Lotz
Abstract:Medical foundation models improve generalization when training AI models with limited labeled data, but remain confined to a single specialty, such as pathology or radiology, and to either sparse or dense outputs, such as classification or segmentation. Here, we present CoM$^3$eT (Co-representation Multidimensional Multitask Medical Transformer), a medical vision foundation model that unifies pathology and radiology, sparse and dense predictions, and two- and higher-dimensional inputs by modeling multidimensional context with attention. CoM$^3$eT outperformed other medical foundation models in an open competition spanning five tomographic, four whole-specimen, and three two-dimensional datasets, covering sparse and dense prediction tasks as well as report generation. When adapted across diverse clinical applications, training fewer than 2.5% of parameters achieved performance comparable to full fine-tuning, enabling research without access to high-performance GPU clusters. Applied to federated learning across hospitals, this approach achieved performance comparable to pooled-data training over internet connections and with consumer-grade hardware.
Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG)
arXiv:2608.16268 [cs.CV]
(or arXiv:2608.16268v1 [cs.CV] for this version)
https://doi.org/10.48550/arXiv.2608.16268 arXiv-issued DOI via DataCite (pending registration)
Text integral sub licența deschisă a publicației · 239 cuvinte
Subiecte conexe
Biomedical Imaging's Real Bottleneck Is the Data, Not the Model
Databricks · 8 oct. 2026
[AINews] Reality Checks on AI News (Yegge shuts down Gas Town, Databricks’ +60% Astra cost)
Latent Space · 17 sept. 2026
News
Quantum Computing Report · 7 iun. 2026
FOCUS: Benchmarking Retinal Model Generalization from Foundation Vision Encoders to Multimodal LLMs
arXiv · 29 sept. 2026