aiqbench analysis
P1SCO introduces a novel dataset of social media comments annotated across ten dimensions, enriched with detailed annotator metadata such as demographics and political affiliation. This resource enables researchers to investigate how individual differences and platform dynamics influence social perception and discourse analysis. The dataset's granular structure supports nuanced studies of inter-annotator agreement and the impact of personality traits on interpretation.
P1SCO: Social Dimensions from a Perspectivist Lens
arXiv:2605.25312v2 Announce Type: replace Abstract: We introduce P1SCO, a dataset of social media comments collected from three distinct platforms, annotated according to ten social dimensions to capture the diversity of social interactions and perceptions. The dataset is carefully disaggregated to allow analysis at the level of individual comments, annotators, and platforms. In addition to the social dimension labels, we include rich metadata on the annotators, including demographics, Big Five personality profiles, and political affiliation. This combination of comment-level annotations and annotator-level features enables nuanced analyses of how social perception varies across platforms, individual differences, and demographic factors. By preserving the diversity of annotator perspectives, our dataset supports studies of inter- and intra-annotator agreement, the influence of personality and political orientation on social interpretation, and the cross-platform dynamics of social discourse.
Abstract:We introduce P1SCO, a dataset of social media comments collected from three distinct platforms, annotated according to ten social dimensions to capture the diversity of social interactions and perceptions. The dataset is carefully disaggregated to allow analysis at the level of individual comments, annotators, and platforms. In addition to the social dimension labels, we include rich metadata on the annotators, including demographics, Big Five personality profiles, and political affiliation. This combination of comment-level annotations and annotator-level features enables nuanced analyses of how social perception varies across platforms, individual differences, and demographic factors. By preserving the diversity of annotator perspectives, our dataset supports studies of inter- and intra-annotator agreement, the influence of personality and political orientation on social interpretation, and the cross-platform dynamics of social discourse.
Computation and Language (cs.CL)
arXiv:2605.25312 [cs.CL]
(or arXiv:2605.25312v2 [cs.CL] for this version)
https://doi.org/10.48550/arXiv.2605.25312 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 218 words
Related coverage
Links #5: 2026/07
LessWrong · Jul 31, 2026
Learning Perspectivist Social Meaning via Demographic-Conditioned Fusion Embeddings
arXiv · Jun 8, 2026
From Individuals to Interactions: Benchmarking Gender Bias in Multimodal Large Language Models from the Lens of Social Relationship
arXiv · Oct 15, 2026

Game Design Creativity evals
LessWrong · Oct 12, 2026