Evaluating the Effectiveness of Persona Simulation in Opinion Prediction with GPT-4.1

Title:Evaluating the Effectiveness of Persona Simulation in Opinion Prediction with GPT-4.1

Abstract:Persona simulation involves utilizing large language models (LLMs) to anticipate human choices or interactions based on specific characteristic information. To further understand current limitations and future directions, we tested persona simulation in opinion prediction with GPT-4.1 (knowledge cutoff by June 2024). Using personas from nine U.S. states provided by Columbia University's Personas dataset, GPT-4.1 accurately predicted 2024 election outcomes in eight out of the nine states, only failing in one of the swing states. We then focused on opinions related to medicine and healthcare. With the American Trends Panel Wave 123 dataset from Pew Research Center, GPT-4.1 was able to anticipate beliefs about childhood vaccines with an accuracy of up to 0.94. Furthermore, we applied GPT-4.1 to generate conversations among personas and observed that the simulated dialogues and opinions adhered well to personas' personalities and backgrounds, albeit lacking natural human-like flow. Persona simulation proves to be a promising application of artificial intelligence as long as biases are addressed. In the near future, it will be beneficial to apply it to opinion analysis and reaction prediction in diverse fields ranging from public health to lawmaking to economics.

Computation and Language (cs.CL)

arXiv:2607.20589 [cs.CL]

(or arXiv:2607.20589v1 [cs.CL] for this version)

https://doi.org/10.48550/arXiv.2607.20589 Focus to learn more arXiv-issued DOI via DataCite (pending registration)

Proceedings of the 2025 IEEE International Conference on Data Mining Workshops (ICDMW), pp. 2938-2942

https://doi.org/10.1109/ICDMW69685.2025.00377 Focus to learn more DOI(s) linking to related resources

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.