analiza aiqbench
This article presents Morpheus, a specialized NLP pipeline designed for de-identifying Greek clinical text, addressing a critical gap in healthcare data privacy for non-English languages. By leveraging frontier LLMs to generate synthetic benchmarks and employing a deterministic, local architecture, the study establishes new standards for evaluating recall and over-redaction in low-resource language settings. The findings are significant for AI researchers developing domain-specific models that must operate within strict regulatory frameworks without relying on external cloud APIs.
Morpheus: a fully local, defense-in-depth de-identification engine for Greek clinical text - a benchmark evaluation
arXiv:2610.25155v1 Announce Type: new Abstract: Background and Objective: Clinical free text cannot leave the hospital without de-identification, yet no de-identification system or benchmark exists for Greek clinical text. We evaluated Morpheus, a fully local de-identification pipeline for Greek clinical documents, on 500 synthetic discharge summaries. Methods: The notes (300 cardiac surgery, 100 cardiology, 100 nephrology) were generated by a frontier large language model (Gemini 3.1 Pro) constrained to authentic Greek hospital document layouts. Notes were pre-annotated at generation; two clinicians then independently reviewed and corrected every note, blinded to each other, with disagreements arbitrated. Primary outcomes were recall over must-redact identifiers and over-redaction of clinical content, measured over must-survive probes. Secondary outcomes: residual leaks flagged by an independent egress-review layer, processing time and reproducibility. Results: Inter-annotator agreement was F1 = 0.998. The de-identification rate was 96.59% (7,666/7,937 identifier instances; 95% CI 96.16-96.96): 98.49% for direct and 93.42% for quasi-identifiers. No patient's own complete identifier survived in any note; of 271 residual instances, 199 (73.4%) concerned institutional identity, chiefly the hospital letterhead. The over-redaction rate was 2.82% (190/6,747 probes; 95% CI 2.45-3.24). The egress guard flagged 2 of 271 residuals, both complete telephone numbers. Two independent runs gave identical verdicts. Conclusions: A fully local, deterministic pipeline removed 98.5% of direct patient identifiers from Greek clinical text while altering under 3% of protected clinical content; most residuals concerned the institution, not the patient. To our knowledge this is the first benchmark evaluation of clinical-text de-identification for Greek and the first quantified evaluation of an independent egress-guard layer.
Authors:Filippos-Paschalis Rorris (1 and 2), Nikoleta Stanitsa (3), Tatiana Rorri (1), Foteini Rorri (4), Constantine N. Antonopoulos (5) ((1) Onassis Hospital, Athens, Greece, (2) Great Ormond Street Hospital for Children, London, United Kingdom, (3) Evangelismos General Hospital, Athens, Greece, (4) Universita della Svizzera italiana, Lugano, Switzerland, (5) Attikon University Hospital, University of Athens Medical School, Athens, Greece)
Abstract:Background and Objective: Clinical free text cannot leave the hospital without de-identification, yet no de-identification system or benchmark exists for Greek clinical text. We evaluated Morpheus, a fully local de-identification pipeline for Greek clinical documents, on 500 synthetic discharge summaries. Methods: The notes (300 cardiac surgery, 100 cardiology, 100 nephrology) were generated by a frontier large language model (Gemini 3.1 Pro) constrained to authentic Greek hospital document layouts. Notes were pre-annotated at generation; two clinicians then independently reviewed and corrected every note, blinded to each other, with disagreements arbitrated. Primary outcomes were recall over must-redact identifiers and over-redaction of clinical content, measured over must-survive probes. Secondary outcomes: residual leaks flagged by an independent egress-review layer, processing time and reproducibility. Results: Inter-annotator agreement was F1 = 0.998. The de-identification rate was 96.59% (7,666/7,937 identifier instances; 95% CI 96.16-96.96): 98.49% for direct and 93.42% for quasi-identifiers. No patient's own complete identifier survived in any note; of 271 residual instances, 199 (73.4%) concerned institutional identity, chiefly the hospital letterhead. The over-redaction rate was 2.82% (190/6,747 probes; 95% CI 2.45-3.24). The egress guard flagged 2 of 271 residuals, both complete telephone numbers. Two independent runs gave identical verdicts. Conclusions: A fully local, deterministic pipeline removed 98.5% of direct patient identifiers from Greek clinical text while altering under 3% of protected clinical content; most residuals concerned the institution, not the patient. To our knowledge this is the first benchmark evaluation of clinical-text de-identification for Greek and the first quantified evaluation of an independent egress-guard layer.
24 pages, 2 figures, 3 tables; supplementary material appended. Benchmark corpus: doi:https://doi.org/10.5281/zenodo.22824894
Computation and Language (cs.CL)
arXiv:2610.25155 [cs.CL]
(or arXiv:2610.25155v1 [cs.CL] for this version)
Text integral sub licența deschisă a publicației · 331 cuvinte