analiza aiqbench
STEREODISCO introduces a novel framework for systematically mapping stereotypical associations within the internal activation spaces of large language models. By probing thousands of semantic axes, this research reveals significant divergences between LLM-encoded biases and those documented in traditional social psychology while identifying previously unexplored stereotype dimensions. This work advances our understanding of how foundational model architectures encode societal prejudices beyond standard demographic categories.
STEREODISCO: Discovering Stereotypicality in LLMs
arXiv:2607.27824v1 Announce Type: cross Abstract: LLMs encode, convey, and perpetuate stereotypes. Prior computational research focuses on a small set of semantic axes investigated in social psychology, and operates on word embeddings produced by language models, leaving open which other semantic axes carry stereotypical associations in LLMs and how LLMs internally represent such axes. We introduce STEREODISCO, a framework that adapts the semantic differential method (Osgood et al., 1957) to the systematic study of stereotypes in LLM internal representations. STEREODISCO constructs approx. 2,000 candidate semantic axes from WordNet antonym synsets, recovers each as a geometric axis in the LLM's activation space via probing, and identifies stereotypical axes via a statistical test over concept projections. As a case study, we apply STEREODISCO to social group stereotypes with LLAMA-3-8B-INSTRUCT and MISTRAL-7B-INSTRUCT. We find that the two LLMs agree with each other on social group ratings more than with humans, suggesting that LLM-encoded stereotype content diverges from that documented in social psychology. We also discover stereotypical axes not investigated in prior work -- including humble vs. proud, narrow-minded vs. broad-minded, and cowardly vs. brave, which human annotators independently confirm.
Abstract:LLMs encode, convey, and perpetuate stereotypes. Prior computational research focuses on a small set of semantic axes investigated in social psychology, and operates on word embeddings produced by language models, leaving open which other semantic axes carry stereotypical associations in LLMs and how LLMs internally represent such axes. We introduce STEREODISCO, a framework that adapts the semantic differential method (Osgood et al., 1957) to the systematic study of stereotypes in LLM internal representations. STEREODISCO constructs approx. 2,000 candidate semantic axes from WordNet antonym synsets, recovers each as a geometric axis in the LLM's activation space via probing, and identifies stereotypical axes via a statistical test over concept projections. As a case study, we apply STEREODISCO to social group stereotypes with LLAMA-3-8B-INSTRUCT and MISTRAL-7B-INSTRUCT. We find that the two LLMs agree with each other on social group ratings more than with humans, suggesting that LLM-encoded stereotype content diverges from that documented in social psychology. We also discover stereotypical axes not investigated in prior work -- including humble vs. proud, narrow-minded vs. broad-minded, and cowardly vs. brave, which human annotators independently confirm.
Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
arXiv:2607.27824 [cs.AI]
(or arXiv:2607.27824v1 [cs.AI] for this version)
https://doi.org/10.48550/arXiv.2607.27824 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 277 cuvinte
Subiecte conexe
BenchMIRT: What are LLM benchmarks actually measuring?
Hugging Face · 1 sept. 2026
Personalization Without Stereotyping: Do Speech LLMs Know When to Turn a Deaf Ear?
arXiv · 12 oct. 2026
Evaluating Regional Bias in LLMs From Abstract Stereotype to Concrete Social Decision-Making
arXiv · 30 iul. 2026
How I Turned AI to the Dark Side
IEEE Spectrum · 14 iul. 2026