analiza aiqbench
This study reveals that LLMs in pharmacology often rely on morphological affixes rather than semantic understanding, leading to dangerous overgeneralization where fictitious drug names trigger plausible clinical responses. The research identifies this behavior as a subtle safety risk rooted in early-mid model layers, highlighting a critical gap between model confidence and actual drug-specific knowledge. For developers and safety researchers, this underscores the need for mechanisms that distinguish genuine pharmacological properties from mere linguistic pattern matching in high-stakes domains.
What's in a Name? Morphological Shortcuts by LLMs in Pharmacology
arXiv:2606.05616v1 Announce Type: new Abstract: The morphological form of a word can often give cues to its meaning, but purely relying on these mappings can lead to overgeneralization in high-stakes domains. In the medical domain, for instance, LLMs can confidently reason about fictitious drugs from their affixes alone (e.g., wugcillin) and generate plausible-looking clinical content. We present a behavioral and mechanistic study of LLM "affix heuristics" in pharmacology. Using fictitious drug names built from real affixes, we show that affix signals alone elicit class-level pharmacological responses. We introduce a framework for identifying whether a model's drug semantics are driven mainly by the affix, the stem, or the drug name as a whole. Applied across 653 drugs, our framework reveals that models often induce drug meaning primarily through affix cues, yet rarely explicitly indicate this reliance, and sometimes incorrectly conflate properties among affix-sharing drugs. Activation patching across models further localizes this behavior to early-mid layers. These findings show that morphological shortcuts pose a subtle but measurable risk to safety.
Title:What's in a Name? Morphological Shortcuts by LLMs in Pharmacology
Abstract:The morphological form of a word can often give cues to its meaning, but purely relying on these mappings can lead to overgeneralization in high-stakes domains. In the medical domain, for instance, LLMs can confidently reason about fictitious drugs from their affixes alone (e.g., wugcillin) and generate plausible-looking clinical content. We present a behavioral and mechanistic study of LLM "affix heuristics" in pharmacology. Using fictitious drug names built from real affixes, we show that affix signals alone elicit class-level pharmacological responses. We introduce a framework for identifying whether a model's drug semantics are driven mainly by the affix, the stem, or the drug name as a whole. Applied across 653 drugs, our framework reveals that models often induce drug meaning primarily through affix cues, yet rarely explicitly indicate this reliance, and sometimes incorrectly conflate properties among affix-sharing drugs. Activation patching across models further localizes this behavior to early-mid layers. These findings show that morphological shortcuts pose a subtle but measurable risk to safety.
Computation and Language (cs.CL)
arXiv:2606.05616 [cs.CL]
(or arXiv:2606.05616v1 [cs.CL] for this version)
https://doi.org/10.48550/arXiv.2606.05616 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 268 cuvinte
Subiecte conexe
What's in a Name? Morphological Shortcuts by LLMs in Pharmacology
arXiv · 3 sept. 2026
Why You Shouldn’t Always Trust LLMs as Judges: Understanding Bias in Automated Evaluation
Analytics Vidhya · 12 aug. 2026
Beyond Surface Cues: Disentangling Sociocultural Signals in Multilingual LLMs
arXiv · 25 aug. 2026
Multimodal LLMs Outperform Pathology Foundation Models in Cross-Domain Histological Similarity
arXiv · 29 sept. 2026