analiza aiqbench
This study reveals that widespread LLM usage systematically neutralizes human writing, eroding authorial voice and subtly shifting semantic meaning even when models are instructed to make only grammatical edits. The research further highlights a critical misalignment in scientific peer review, where AI-generated assessments prioritize inflated scores over the clarity and significance of research findings. These results underscore an urgent need to address how large-scale model adoption is reshaping cultural and institutional communication standards.
How LLMs Distort Our Written Language
arXiv:2603.18161v2 Announce Type: replace-cross Abstract: Large language models (LLMs) are used by over a billion people globally, most often to assist with writing. In this work, we demonstrate that LLMs not only alter the voice and tone of human writing but also consistently alter the intended meaning. First, we conduct a human user study to understand how people actually interact with LLMs when using them for writing. Our findings reveal that extensive LLM use led to a nearly 70% increase in essays that remained neutral in answering the topic question. Significantly more heavy LLM users reported that the writing was less creative and not in their voice. Next, using a dataset of human-written essays that was collected in 2021 before the widespread release of LLMs, we study how asking an LLM to revise the essay based on the human-written feedback in the dataset induces large changes in the resulting content and meaning. We find that even when LLMs are prompted with expert feedback and asked to only make grammar edits, they still change the text in a way that significantly alters its semantic meaning. We then examine LLM-generated text in the wild, specifically focusing on the 21% of AI-generated scientific peer reviews at a recent top AI conference. We find that LLM-generated reviews place significantly less weight on clarity and significance of the research, and assign scores that, on average, are a full point higher. These findings highlight a misalignment between the perceived benefit of AI use and an implicit, consistent effect on the semantics of human writing, motivating future work on how widespread AI writing will affect our cultural and scientific institutions.
Abstract:Large language models (LLMs) are used by over a billion people globally, most often to assist with writing. In this work, we demonstrate that LLMs not only alter the voice and tone of human writing but also consistently alter the intended meaning. First, we conduct a human user study to understand how people actually interact with LLMs when using them for writing. Our findings reveal that extensive LLM use led to a nearly 70% increase in essays that remained neutral in answering the topic question. Significantly more heavy LLM users reported that the writing was less creative and not in their voice. Next, using a dataset of human-written essays that was collected in 2021 before the widespread release of LLMs, we study how asking an LLM to revise the essay based on the human-written feedback in the dataset induces large changes in the resulting content and meaning. We find that even when LLMs are prompted with expert feedback and asked to only make grammar edits, they still change the text in a way that significantly alters its semantic meaning. We then examine LLM-generated text in the wild, specifically focusing on the 21% of AI-generated scientific peer reviews at a recent top AI conference. We find that LLM-generated reviews place significantly less weight on clarity and significance of the research, and assign scores that, on average, are a full point higher. These findings highlight a misalignment between the perceived benefit of AI use and an implicit, consistent effect on the semantics of human writing, motivating future work on how widespread AI writing will affect our cultural and scientific institutions.
Computation and Language (cs.CL); Artificial Intelligence (cs.AI)
arXiv:2603.18161 [cs.CL]
(or arXiv:2603.18161v2 [cs.CL] for this version)
https://doi.org/10.48550/arXiv.2603.18161 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 364 cuvinte
Subiecte conexe
Script Fragmentation and Format: What Drives the English-Bengali Performance Gap in Open LLMs?
arXiv · 16 sept. 2026
Reviewing failure modes in science as we prepare for AI researchers
LessWrong · 8 oct. 2026
Closing the Binary Blind Spot: Domain-Knowledge Injection for LLM Malware Triage in npm Packages
Databricks · 16 oct. 2026
Distributional Open-Ended Evaluation of LLM Cultural Value Alignment Based on Value Codebook
arXiv · 9 iun. 2026