aiqbench analysis
This study demonstrates a practical application of weakly supervised NLP for extracting diagnoses from unstructured clinical text, significantly reducing the manual annotation burden required for large-scale epidemiological research. By leveraging transformer-based embeddings and clustering techniques on Italian discharge letters, the model achieves performance comparable to fully supervised approaches while saving over 1,500 hours of labeling time. The methodology offers a scalable solution for automating cohort selection in medical AI pipelines without relying on extensive ground truth data.
Automatic identification of diagnosis from hospital discharge letters via weakly supervised Natural Language Processing
arXiv:2410.15051v3 Announce Type: replace-cross Abstract: Identifying patient diagnoses from hospital discharge letters is essential for large-scale cohort selection and epidemiological research, but traditional supervised approaches require extensive manual annotation, which is often impractical for large textual datasets. We present a weakly supervised Natural Language Processing (NLP) pipeline for classifying Italian discharge letters without document-level manual annotation. The method extracts diagnosis-related sentences, generates semantic embeddings using a transformer model further pre-trained on Italian medical documents, and applies a two-level clustering procedure to derive weak labels that are then used to train a document-level classifier. The approach was evaluated in a case study on bronchiolitis using 33,176 discharge letters of children admitted to 44 emergency rooms or hospitals in the Veneto Region, Italy, between 2017 and 2020. The best weakly supervised model achieved an AUROC of 77.68% ($\pm4.30\%$), an AUPRC of 73.13% ($\pm4.93\%$), and an F1-score of 78.14% ($\pm4.89\%$) against manually annotated data. Performance surpassed unsupervised baselines and approached fully supervised models, while reducing the need for manual annotation by more than 1,500 hours for a dataset of this size. Similar model rankings were observed in a secondary validation on a smaller bronchitis dataset (3,188 discharge letters, 2020-2025), where the best weakly supervised model achieved an AUPRC of 76.72% ($\pm 5.02\%$). These results suggest the potential of weakly supervised NLP methods for scalable disease identification from clinical discharge letters.
Abstract:Identifying patient diagnoses from hospital discharge letters is essential for large-scale cohort selection and epidemiological research, but traditional supervised approaches require extensive manual annotation, which is often impractical for large textual datasets. We present a weakly supervised Natural Language Processing (NLP) pipeline for classifying Italian discharge letters without document-level manual annotation. The method extracts diagnosis-related sentences, generates semantic embeddings using a transformer model further pre-trained on Italian medical documents, and applies a two-level clustering procedure to derive weak labels that are then used to train a document-level classifier. The approach was evaluated in a case study on bronchiolitis using 33,176 discharge letters of children admitted to 44 emergency rooms or hospitals in the Veneto Region, Italy, between 2017 and 2020. The best weakly supervised model achieved an AUROC of 77.68% ($\pm4.30\%$), an AUPRC of 73.13% ($\pm4.93\%$), and an F1-score of 78.14% ($\pm4.89\%$) against manually annotated data. Performance surpassed unsupervised baselines and approached fully supervised models, while reducing the need for manual annotation by more than 1,500 hours for a dataset of this size. Similar model rankings were observed in a secondary validation on a smaller bronchitis dataset (3,188 discharge letters, 2020-2025), where the best weakly supervised model achieved an AUPRC of 76.72% ($\pm 5.02\%$). These results suggest the potential of weakly supervised NLP methods for scalable disease identification from clinical discharge letters.
Computation and Language (cs.CL); Machine Learning (cs.LG)
arXiv:2410.15051 [cs.CL]
(or arXiv:2410.15051v3 [cs.CL] for this version)
https://doi.org/10.1038/s41598-026-56721-0 DOI(s) linking to related resources
Full text under the publisher's open licence · 243 words
Related coverage
On Sleep-Wake Disorder Identification from Polysomnographic Representations
arXiv · Oct 14, 2026
BiMamba2 Masked Discrete-Unit Prediction for Multilingual Speech Representation for Unsupervised Speech in the Wild Challenge
arXiv · Sep 25, 2026
WildMatch: Weakly Supervised Image Matcher Adaptation for Wildlife Re-Identification
arXiv · Oct 7, 2026
What Should We Ask Next? Retrieval-Aware Question Learning under Partial Evidence
arXiv · Sep 22, 2026