Modelling Adjectival Modification Effects on Semantic Plausibility

Abstract:While the task of assessing the plausibility of events such as "news is relevant" has been addressed by a growing body of work, less attention has been paid to capturing changes in plausibility as triggered by event modification. Understanding changes in plausibility is relevant for tasks such as dialogue generation, commonsense reasoning, and hallucination detection, as it allows to correctly model, for example, "false news is relevant", which is of lower relevance but higher concern due to potential disinformation. In this work, we tackle the Adept challenge benchmark (Emami et al. 2021) consisting of 16K English sentence pairs differing by exactly one adjectival modifier (e.g., false.) Our modeling experiments provide a conceptually novel method using sentence transformers and reveal that sentence transformers struggle despite their conceptual alignment with the task at hand, underperforming in comparison to transformers like RoBERTa. Finally, we discuss our findings in relation to prior work and present a detailed error analysis to shed light on potential sources for ST underperformance, highlighting advantages and shortcomings of the examined methods for balancing out train and test data.

Computation and Language (cs.CL)

arXiv:2507.21828 [cs.CL]

(or arXiv:2507.21828v2 [cs.CL] for this version)

https://doi.org/10.48550/arXiv.2507.21828 Focus to learn more arXiv-issued DOI via DataCite

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.