analiza aiqbench
This research introduces MIFR, a framework that fuses clinical photos with dermoscopic images using Vision Transformer encoders to address both diagnostic accuracy and fairness across diverse skin tones. By employing a multi-objective loss function, the model aligns embeddings from different modalities while explicitly penalizing performance disparities based on skin type, achieving competitive results on standard datasets. The work advances medical AI by demonstrating that joint representation learning can simultaneously improve predictive power and reduce algorithmic bias in dermatological applications.
MIFR: A Modality-Invariant and Fair Representation Framework for Skin Disease Classification
arXiv:2608.18774v1 Announce Type: cross Abstract: Skin diseases represent a major global public health burden, yet machine learning tools developed to assist in their diagnosis suffer from two critical limitations: reliance on only one modality for diagnosis and systematic performance disparities across skin tones. While existing approaches address each challenge separately, this work proposes a modality-invariant framework with fair representation (MIFR) for skin disease classification. The architecture pairs clinical photographs with dermoscopic images using ViT-based encoders, projecting each input into a high-dimensional embedding space via modality-specific projection heads. The resulting model is trained with a five-component multi-objective loss including weighted cross-entropy for classification, confusion and skin-type classification losses for fairness, per-modality supervised contrastive loss for class alignment, and a modality-invariance loss for clinical and dermoscopic modality alignment. Experiments on the HIBA+Derm7pt paired dataset and the external PAD-UFES-20 and ISIC 2019 datasets showed that modality-invariant representation learning provides competitive predictive performance compare to relevant baseline models and competitive fairness on the internal dataset. t-SNE visualizations confirmed that clinical and dermoscopic embeddings of the same disease are geometrically aligned, validating the joint objectives.
Abstract:Skin diseases represent a major global public health burden, yet machine learning tools developed to assist in their diagnosis suffer from two critical limitations: reliance on only one modality for diagnosis and systematic performance disparities across skin tones. While existing approaches address each challenge separately, this work proposes a modality-invariant framework with fair representation (MIFR) for skin disease classification. The architecture pairs clinical photographs with dermoscopic images using ViT-based encoders, projecting each input into a high-dimensional embedding space via modality-specific projection heads. The resulting model is trained with a five-component multi-objective loss including weighted cross-entropy for classification, confusion and skin-type classification losses for fairness, per-modality supervised contrastive loss for class alignment, and a modality-invariance loss for clinical and dermoscopic modality alignment. Experiments on the HIBA+Derm7pt paired dataset and the external PAD-UFES-20 and ISIC 2019 datasets showed that modality-invariant representation learning provides competitive predictive performance compare to relevant baseline models and competitive fairness on the internal dataset. t-SNE visualizations confirmed that clinical and dermoscopic embeddings of the same disease are geometrically aligned, validating the joint objectives.
Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG)
arXiv:2608.18774 [cs.CV]
(or arXiv:2608.18774v1 [cs.CV] for this version)
https://doi.org/10.48550/arXiv.2608.18774 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 275 cuvinte
Subiecte conexe
BRICS Draws a New Global Line: Full Text of New Delhi Declaration
slguardian.org · 12 sept. 2026
FairSSL: Fair Multimodal Self-Supervised Learning
arXiv · 2 oct. 2026
Macao Polytechnic Team Detects Angular Order in Complex Images
Quantum Zeitgeist · 19 aug. 2026
FRAME: separating sampling variation from representational cause in medical imaging fairness
arXiv · 27 aug. 2026