aiqbench analysis
This study reveals that machine learning models for tuberculosis screening via cough acoustics fail to generalize across datasets, performing significantly worse on external data than within-dataset benchmarks. The research identifies recording device artifacts and acquisition-specific variability as primary drivers of poor transferability, outperforming population shifts in degrading model utility. Consequently, the authors conclude that rigorous external validation is mandatory before such clinical AI tools can be considered ready for deployment.
Why ML-based cough models do not generalize: a systematic cross-dataset evaluation for tuberculosis screening
arXiv:2608.25846v1 Announce Type: cross Abstract: Cough acoustics are promising for non-invasive tuberculosis (TB) screening, yet whether machine learning (ML) models capture disease-related acoustics or artifacts of data collection remains unresolved. We evaluated the cross-dataset generalizability of classical ML and deep learning (DL) cough-based TB classifiers across three independent datasets. Despite moderate within-dataset performance (ROC-AUC up to $0.755 \pm 0.056$), both pipelines fail to generalize, with external performance frequently below 0.6, indicating a possible limitation of the data. We further observed audio representations are organized by recording device and dataset rather than TB status, predicted TB probability tracks country-level prevalence in CODA, and device mismatch degrades transfer while device-diverse training improves it. Additionally, a clinical-variable baseline generalizes more consistently (ROC-AUC $0.655 - 0.711$), indicating acquisition-specific variability is a stronger driver of poor generalizability than population shift. High within-dataset performance is not enough. External validation is essential before cough-based TB models are clinically ready.
Abstract:Cough acoustics are promising for non-invasive tuberculosis (TB) screening, yet whether machine learning (ML) models capture disease-related acoustics or artifacts of data collection remains unresolved. We evaluated the cross-dataset generalizability of classical ML and deep learning (DL) cough-based TB classifiers across three independent datasets. Despite moderate within-dataset performance (ROC-AUC up to $0.755 \pm 0.056$), both pipelines fail to generalize, with external performance frequently below 0.6, indicating a possible limitation of the data. We further observed audio representations are organized by recording device and dataset rather than TB status, predicted TB probability tracks country-level prevalence in CODA, and device mismatch degrades transfer while device-diverse training improves it. Additionally, a clinical-variable baseline generalizes more consistently (ROC-AUC $0.655 - 0.711$), indicating acquisition-specific variability is a stronger driver of poor generalizability than population shift. High within-dataset performance is not enough. External validation is essential before cough-based TB models are clinically ready.
Audio and Speech Processing (eess.AS); Artificial Intelligence (cs.AI); Machine Learning (cs.LG); Sound (cs.SD); Signal Processing (eess.SP)
arXiv:2608.25846 [eess.AS]
(or arXiv:2608.25846v1 [eess.AS] for this version)
https://doi.org/10.48550/arXiv.2608.25846 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 254 words
Related coverage
Concrete Evaluations to Investigate the OpenAI Model That Hacked Hugging Face
AI Alignment Forum · Aug 3, 2026
What We Learned Trying to Catch AI Liars: An Aletheia's Quest Retrospective
EleutherAI · Aug 25, 2026
A Multi-Domain Feature Fusion Framework for Generalizable Deepfake Detection Across Different Generators
arXiv · Jun 15, 2026
Manifold Steering from a Natural State is Effective and Generalizable, but Relies on Model Accuracy
LessWrong · Oct 12, 2026