aiqbench analysis
This study demonstrates that robustness-oriented training strategies, specifically combining entity masking with domain-adversarial techniques, can effectively mitigate the trade-off between in-domain accuracy and robustness against distributional shifts in biomedical literature classification. By selectively suppressing spurious topical correlations while preserving methodological signals, the approach ensures models rely on explicit cues rather than superficial lexical patterns. This highlights the critical need for feature-level robustness analysis when deploying language models for evidence synthesis in dynamic research environments.
Robust Biomedical Publication Type and Study Design Classification with Knowledge-Guided Perturbations
arXiv:2605.11502v2 Announce Type: replace Abstract: Accurately and consistently indexing biomedical literature by publication type and study design is essential for supporting evidence synthesis and knowledge discovery. Prior work on automated publication type and study design indexing has primarily focused on expanding label coverage, enriching feature representations, and improving in-domain accuracy, with evaluation typically conducted on data drawn from the same distribution as training. Although pretrained biomedical language models achieve strong performance under these settings, models optimized for in-domain accuracy may rely on superficial lexical or dataset-specific cues, resulting in reduced robustness under distributional shift. In this study, we introduce an evaluation framework based on controlled semantic perturbations to assess the robustness of a publication type classifier and investigate robustness-oriented training strategies that combine entity masking and domain-adversarial training to mitigate reliance on spurious topical correlations. Our results show that the commonly observed trade-off between robustness and in-domain accuracy can be mitigated when robustness objectives are designed to selectively suppress non-task-defining features while preserving salient methodological signals. We find that these improvements arise from two complementary mechanisms: (1) increased reliance on explicit methodological cues when such cues are present in the input, and (2) reduced reliance on spurious domain-specific topical features. These findings highlight the importance of feature-level robustness analysis for publication type and study design classification and suggest that refining masking and adversarial objectives to more selectively suppress topical information may further improve robustness. Data, code, and models are available at: https://github.com/ScienceNLP-Lab/MultiTagger-v2/tree/main/ICHI
Abstract:Accurately and consistently indexing biomedical literature by publication type and study design is essential for supporting evidence synthesis and knowledge discovery. Prior work on automated publication type and study design indexing has primarily focused on expanding label coverage, enriching feature representations, and improving in-domain accuracy, with evaluation typically conducted on data drawn from the same distribution as training. Although pretrained biomedical language models achieve strong performance under these settings, models optimized for in-domain accuracy may rely on superficial lexical or dataset-specific cues, resulting in reduced robustness under distributional shift. In this study, we introduce an evaluation framework based on controlled semantic perturbations to assess the robustness of a publication type classifier and investigate robustness-oriented training strategies that combine entity masking and domain-adversarial training to mitigate reliance on spurious topical correlations. Our results show that the commonly observed trade-off between robustness and in-domain accuracy can be mitigated when robustness objectives are designed to selectively suppress non-task-defining features while preserving salient methodological signals. We find that these improvements arise from two complementary mechanisms: (1) increased reliance on explicit methodological cues when such cues are present in the input, and (2) reduced reliance on spurious domain-specific topical features. These findings highlight the importance of feature-level robustness analysis for publication type and study design classification and suggest that refining masking and adversarial objectives to more selectively suppress topical information may further improve robustness. Data, code, and models are available at: this https URL
Computation and Language (cs.CL)
arXiv:2605.11502 [cs.CL]
(or arXiv:2605.11502v2 [cs.CL] for this version)
https://doi.org/10.48550/arXiv.2605.11502 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 332 words
Related coverage
Article: Governing AI in the Cloud: A Practical Guide for Architects
InfoQ · Jun 15, 2026
A Standardized Framework for Machine Learning in Power System Protection
arXiv · Aug 21, 2026
Leakage-Safe Empirical Benchmarking of EEG-Based Machine Learning Pipelines for Dementia Classification
arXiv · Sep 22, 2026
Design and Validation of a Lightweight 1D CNN for Affective Touch Classification in Soft Plush Companions
arXiv · Jul 21, 2026