aiqbench analysis
Semi-MedRef introduces a novel semi-supervised framework for medical referring image segmentation that maintains robust cross-modal alignment despite aggressive data augmentations like CutMix. By integrating position-aware text regularization and anatomically weighted contrastive learning, the method effectively leverages unlabeled datasets to outperform existing supervised approaches on critical lesion detection tasks. This advancement addresses the high annotation costs inherent in medical imaging while preserving the fidelity required for precise image-text correspondence.
Semi-MedRef: Semi-Supervised Medical Referring Image Segmentation with Cross-Modal Alignment
arXiv:2605.15720v2 Announce Type: replace-cross Abstract: Medical referring image segmentation (MRIS) predicts lesion masks from medical images and natural-language referring expressions, but acquiring paired pixel-level annotations and referring texts is costly. Semi-supervised learning (SSL) can alleviate this burden by exploiting unlabeled data, yet its effectiveness depends on preserving image--text alignment under strong perturbations. Existing SSL methods for referring segmentation rely on independent or simple multimodal perturbations (e.g., left--right flips), while stronger augmentations such as CutMix remain largely unexplored because they can disrupt cross-modal correspondence. We propose Semi-MedRef, a teacher--student SSL framework that explicitly preserves alignment between medical images and positional language through three complementary components: T-PatchMix, an alignment-preserving cross-modal augmentation that synchronizes patch mixing with positional-language and pseudo-mask updates; PosAug, a position-aware text augmentation that regularizes reliance on positional expressions; and Positional Affinity Contrastive Learning (PACL), which exploits coarse positional cues to construct region-aware supervision through anatomically weighted soft positives, encouraging anatomically grounded cross-modal representation learning. Experiments on QaTa-COV19 and MosMedData+ demonstrate that Semi-MedRef consistently outperforms state-of-the-art fully supervised and semi-supervised MRIS methods across all label regimes.
Abstract:Medical referring image segmentation (MRIS) predicts lesion masks from medical images and natural-language referring expressions, but acquiring paired pixel-level annotations and referring texts is costly. Semi-supervised learning (SSL) can alleviate this burden by exploiting unlabeled data, yet its effectiveness depends on preserving image--text alignment under strong perturbations. Existing SSL methods for referring segmentation rely on independent or simple multimodal perturbations (e.g., left--right flips), while stronger augmentations such as CutMix remain largely unexplored because they can disrupt cross-modal correspondence. We propose Semi-MedRef, a teacher--student SSL framework that explicitly preserves alignment between medical images and positional language through three complementary components: T-PatchMix, an alignment-preserving cross-modal augmentation that synchronizes patch mixing with positional-language and pseudo-mask updates; PosAug, a position-aware text augmentation that regularizes reliance on positional expressions; and Positional Affinity Contrastive Learning (PACL), which exploits coarse positional cues to construct region-aware supervision through anatomically weighted soft positives, encouraging anatomically grounded cross-modal representation learning. Experiments on QaTa-COV19 and MosMedData+ demonstrate that Semi-MedRef consistently outperforms state-of-the-art fully supervised and semi-supervised MRIS methods across all label regimes.
Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG)
arXiv:2605.15720 [cs.CV]
(or arXiv:2605.15720v2 [cs.CV] for this version)
https://doi.org/10.48550/arXiv.2605.15720 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 269 words
Related coverage
OFD-Net: Teacher-Free Reliable Semi-supervised Medical Image Segmentation with Orthogonal Feature Disentanglement Net of Foreground-Background
arXiv · Jul 21, 2026
ESRVS: Extreme Semi-Supervised Retinal Vessel Segmentation with a Single Annotated Image
arXiv · Jul 28, 2026
Beyond Visual Cues: CoT-Enhanced Reasoning for Semi-supervised Medical Image Segmentation
arXiv · Jun 17, 2026
SAUF-Net: Structure--Appearance Representation Learning with Uncertainty Feedback for Semi-Supervised Medical Image Segmentation
arXiv · Sep 3, 2026