aiqbench analysis
This arXiv preprint studies how to align LLM output distributions with human response distributions, a property relevant to using models to simulate survey respondents. It reports that alignment transfers to unseen questions within the same domain, but cross-topic generalization depends on how related the topics are, and aligned models can lose capabilities such as creativity. For AI readers, the practical takeaway is that simulated-population results need validation beyond the training domain, since the generalization behavior is described as unstable and hard to predict.
Distributional Alignment of LLMs and Its Generalization Properties
arXiv:2610.31687v1 Announce Type: cross Abstract: Large language models are increasingly used to simulate and probe human behavior, e.g., for surveys. For these applications, a key property is that outputs satisfy distributional alignment: to match its output distribution with some reference (in this case, human) distribution. An obstacle in achieving distributional alignment is generalizing from small training data. In this work, we study (1) how to distributionally align LLMs; and (2) how aligned LLMs might generalize to unseen samples and unrelated topics and tasks. Using two large-scale benchmarks, we find that LLMs can indeed achieve distributional alignment and generalize to unseen questions of the same domain. Cross-topic generalization depends on the relationship between topics (e.g., questions about politics are related to trust in institutions). However, aligned LLMs can degrade in surprising ways (e.g., lose their creative ability), suggesting that its generalization properties are inherently unpredictable and unstable.
Abstract:Large language models are increasingly used to simulate and probe human behavior, e.g., for surveys. For these applications, a key property is that outputs satisfy distributional alignment: to match its output distribution with some reference (in this case, human) distribution. An obstacle in achieving distributional alignment is generalizing from small training data. In this work, we study (1) how to distributionally align LLMs; and (2) how aligned LLMs might generalize to unseen samples and unrelated topics and tasks. Using two large-scale benchmarks, we find that LLMs can indeed achieve distributional alignment and generalize to unseen questions of the same domain. Cross-topic generalization depends on the relationship between topics (e.g., questions about politics are related to trust in institutions). However, aligned LLMs can degrade in surprising ways (e.g., lose their creative ability), suggesting that its generalization properties are inherently unpredictable and unstable.
Computation and Language (cs.CL); Artificial Intelligence (cs.AI)
arXiv:2610.31687 [cs.CL]
(or arXiv:2610.31687v1 [cs.CL] for this version)
https://doi.org/10.48550/arXiv.2610.31687 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 237 words
Related coverage
SoFT: Soft Targets for Generalizable LLM Fine-Tuning
arXiv · Sep 30, 2026
Distribution-Conditioned Transport
arXiv · Sep 28, 2026
Language Guidance for Supervised Vision Training: An Empirical Study of Generalization
arXiv · Oct 21, 2026
Improving Generalization and Robustness in Offline Reinforcement Learning via Boundary-Aware Data Augmentation
arXiv · Sep 18, 2026