aiqbench analysis
This research presents AudioFuse, a hybrid architecture combining Vision Transformers and 1D CNNs to improve phonocardiogram classification by fusing spectral and temporal signal representations. The model demonstrates superior robustness to domain shifts compared to single-modality baselines, offering a practical approach for generalizable biomedical audio analysis without relying on large-scale pre-training.
AudioFuse: Unified Spectral-Temporal Learning via a Hybrid ViT-1D CNN Architecture for Robust Phonocardiogram Classification
arXiv:2509.23454v2 Announce Type: replace-cross Abstract: Biomedical audio signals, such as phonocardiograms (PCG), are inherently rhythmic and contain diagnostic information in both their spectral (tonal) and temporal domains. Standard 2D spectrograms provide rich spectral features but compromise the phase information and temporal precision of the 1D waveform. We propose AudioFuse, an architecture that simultaneously learns from both complementary representations to classify PCGs. To mitigate the overfitting risk common in fusion models, we integrate a custom, wide-and-shallow Vision Transformer (ViT) for spectrograms with a shallow 1D CNN for raw waveforms. On the PhysioNet 2016 dataset, AudioFuse achieves a state-of-the-art competitive ROC-AUC of 0.8608 when trained from scratch, outperforming its spectrogram (0.8066) and waveform (0.8223) baselines. Moreover, it demonstrates superior robustness to domain shift on the challenging PASCAL dataset, maintaining an ROC-AUC of 0.7181 while the spectrogram baseline collapses (0.4873). Fusing complementary representations thus provides a strong inductive bias, enabling the creation of efficient, generalizable classifiers without requiring large-scale pre-training. The complete source code for our baseline and AudioFuse models is publicly available on GitHub at: https://github.com/Saiful185/AudioFuse.
Abstract:Biomedical audio signals, such as phonocardiograms (PCG), are inherently rhythmic and contain diagnostic information in both their spectral (tonal) and temporal domains. Standard 2D spectrograms provide rich spectral features but compromise the phase information and temporal precision of the 1D waveform. We propose AudioFuse, an architecture that simultaneously learns from both complementary representations to classify PCGs. To mitigate the overfitting risk common in fusion models, we integrate a custom, wide-and-shallow Vision Transformer (ViT) for spectrograms with a shallow 1D CNN for raw waveforms. On the PhysioNet 2016 dataset, AudioFuse achieves a state-of-the-art competitive ROC-AUC of 0.8608 when trained from scratch, outperforming its spectrogram (0.8066) and waveform (0.8223) baselines. Moreover, it demonstrates superior robustness to domain shift on the challenging PASCAL dataset, maintaining an ROC-AUC of 0.7181 while the spectrogram baseline collapses (0.4873). Fusing complementary representations thus provides a strong inductive bias, enabling the creation of efficient, generalizable classifiers without requiring large-scale pre-training. The complete source code for our baseline and AudioFuse models is publicly available on GitHub at: this https URL.
Published in the Proceedings of 2026 IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP) main conference. Find the published version at: this https URL
Audio and Speech Processing (eess.AS); Artificial Intelligence (cs.AI); Machine Learning (cs.LG); Sound (cs.SD); Signal Processing (eess.SP)
arXiv:2509.23454 [eess.AS]
(or arXiv:2509.23454v2 [eess.AS] for this version)
M. S. B. Siddiqui and U. Saha, "AudioFuse: Unified Spectral-Temporal Learning Via A Hybrid VIT-1D CNN Architecture for Phonocardiogram Classification," ICASSP 2026, Barcelona, Spain, 2026, pp. 5711-5715
https://doi.org/10.1109/ICASSP55912.2026.11464477 DOI(s) linking to related resources
From: Md. Saiful Bari Siddiqui [view email] [v1] Sat, 27 Sep 2025 18:52:50 UTC (245 KB) [v2] Sat, 5 Sep 2026 19:38:50 UTC (216 KB)
Full text under the publisher's open licence · 279 words
Related coverage
WaVeFuse: Regime-Adaptive Equity Index Forecasting via Channel-Wise Wavelet Denoising and Vertical Attention Fusion
arXiv · Sep 15, 2026
NVIDIA Nemotron 3 Nano Omni Powers Multimodal Agent Reasoning in a Single Efficient Open Model
NVIDIA · Apr 28, 2026
Towards Robust Arabic Speech Emotion Recognition with Deep Learning
arXiv · Jun 10, 2026
Multi-Modal Spatio-Temporal Graph Neural Network with Mixture of Experts for Soil Organic Carbon Prediction
arXiv · Jun 16, 2026