aiqbench analysis
This research demonstrates how Large Language Models can bridge the semantic gap in self-supervised audio encoders, enabling zero-shot classification of respiratory sounds without task-specific labeled data. By synthesizing structured medical reports to create dense semantic anchors, the proposed framework significantly outperforms general-purpose models like CLAP and Qwen2-Audio in clinical diagnostic tasks. The approach validates that targeted semantic alignment is more effective than scaling large general models for specialized medical audio analysis.
Zero-Shot Respiratory Sound Classification through LLM-Augmented Audio-Text Alignment
arXiv:2609.00055v1 Announce Type: cross Abstract: Self-supervised respiratory encoders lack semantic grounding in clinical domain needed for zero-shot inference, limiting their utility without task-specific labeled data. We propose a framework that aligns these encoders with medical terminology in a shared latent space turning them into a zero-shot-capable foundation model. To address paired data scarcity, we use a medical LLM to synthesize structured reports from metadata, creating dense semantic anchors for contrastive learning. Our training combines a sigmoid-based contrastive loss with encoder's native SSL objective and similarity-aware negative sampling to sharpen pathological boundaries. Across 9 tasks on 6 datasets, our method achieves a 61.3% mean zero-shot AUC, surpassing CLAP (51.4%) and Qwen2-Audio (54.9%) while reaching the highest linear probing AUC (71.6%) with only 43% of data used by full-scale baselines, showing that structured semantic alignment outperforms large-scale, general-purpose models in clinical diagnostics.
Abstract:Self-supervised respiratory encoders lack semantic grounding in clinical domain needed for zero-shot inference, limiting their utility without task-specific labeled data. We propose a framework that aligns these encoders with medical terminology in a shared latent space turning them into a zero-shot-capable foundation model. To address paired data scarcity, we use a medical LLM to synthesize structured reports from metadata, creating dense semantic anchors for contrastive learning. Our training combines a sigmoid-based contrastive loss with encoder's native SSL objective and similarity-aware negative sampling to sharpen pathological boundaries. Across 9 tasks on 6 datasets, our method achieves a 61.3% mean zero-shot AUC, surpassing CLAP (51.4%) and Qwen2-Audio (54.9%) while reaching the highest linear probing AUC (71.6%) with only 43% of data used by full-scale baselines, showing that structured semantic alignment outperforms large-scale, general-purpose models in clinical diagnostics.
Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Sound (cs.SD)
arXiv:2609.00055 [cs.CL]
(or arXiv:2609.00055v1 [cs.CL] for this version)
https://doi.org/10.48550/arXiv.2609.00055 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 233 words
Related coverage
A Benchmark of Generative Methods for Zero-Shot Environmental Sound Classification
arXiv · Jul 21, 2026
[AINews] Opus 5.5 is good at explainer videos
Latent Space · Sep 29, 2026
AeroSpectra Sentinel: An Auditable LLM Prompt-Chaining Decision-Support Workflow for Acute Asthma Risk Assessment from Respiratory Sounds and Clinical Signals
arXiv · Jun 9, 2026
Optimizing 2D Input Representations and Sub-phase Fusion Strategies for Differential Diagnosis of Asthma and COPD Using CNN- and GRU-Based Networks
arXiv · Jun 10, 2026