analiza aiqbench
This research investigates the structural limitations of LLMs in summarizing literature, specifically analyzing how models struggle to maintain narrative linearity and uniformity when mapping source chapters to summary sentences. The study highlights a fundamental challenge for natural language generation tasks where artistic expression and plot complexity cause significant deviations from original text structures that even human annotators find difficult to replicate consistently.
The Plot Thins: Uniformity and Linearity in Literary Summaries
arXiv:2608.17218v1 Announce Type: new Abstract: Works of literature are complicated; they balance plot, suspense, surprise, and artistic expression. Summaries of literature prioritize plot, and therefore may deviate from their sources. Using a combination of manual and LLM-based annotation, we construct a dataset mapping sentences from 150 novel summaries to their respective source chapters. We find the task unexpectedly difficult for both human and model annotators. Using the sentence-to-chapter mappings, we then measure summary linearity, the degree to which it maintains the source's order of events, and uniformity, the degree to which a summary spreads attention equally across a source. By examining when and how summaries break linearity and uniformity, we identify differences in how literary works and summaries express plot, particularly with regard to the clarity and prominence with which narrative details are described.
Abstract:Works of literature are complicated; they balance plot, suspense, surprise, and artistic expression. Summaries of literature prioritize plot, and therefore may deviate from their sources. Using a combination of manual and LLM-based annotation, we construct a dataset mapping sentences from 150 novel summaries to their respective source chapters. We find the task unexpectedly difficult for both human and model annotators. Using the sentence-to-chapter mappings, we then measure summary linearity, the degree to which it maintains the source's order of events, and uniformity, the degree to which a summary spreads attention equally across a source. By examining when and how summaries break linearity and uniformity, we identify differences in how literary works and summaries express plot, particularly with regard to the clarity and prominence with which narrative details are described.
Computation and Language (cs.CL)
arXiv:2608.17218 [cs.CL]
(or arXiv:2608.17218v1 [cs.CL] for this version)
https://doi.org/10.48550/arXiv.2608.17218 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 224 cuvinte
Subiecte conexe

Challenge: Hand coding weights for efficient sequence memorisation
AI Alignment Forum · 23 iul. 2026
DI Series: Serving Kimi-K3 MXFP4 Wide-EP Disaggregated on AMD Instinct MI300X and MI325X
AMD · 8 oct. 2026

Manifold Steering from a Natural State is Effective and Generalizable, but Relies on Model Accuracy
LessWrong · 12 oct. 2026
Fable is SOTA at CIFAR Speedrun (& specification gaming)
LessWrong · 20 iul. 2026