analiza aiqbench
This research addresses a critical gap in AI safety by introducing a multilingual benchmark specifically designed to detect and mitigate anti-LGBTQ biases in German and English language models. The study demonstrates that while fine-tuning on progressive content can reduce these stereotypes, the effectiveness varies significantly across different model architectures and specific identities, highlighting the complexity of cultural adaptation in bias mitigation.
Evaluating and Mitigating Anti-LGBTQ Biases in German and Multilingual Language Models
arXiv:2608.30884v1 Announce Type: cross Abstract: While gender and racial biases in language models have been widely studied, anti-LGBTQ biases remain underexplored, particularly beyond English. Existing benchmarks often do not capture cultural and linguistic variation and rely on gender representations. This paper introduces a multilingual German-English benchmark dataset for the evaluation of anti-LGBTQ biases in language models. It combines community-sourced stereotypes from German-speaking queer individuals with a German translation of WinoQueer. The data is used to evaluate eight language models across sizes and architectures and explore mitigation through fine-tuning on community and progressive media content. Results show that language models reproduce anti-queer stereotypes, with variation across identities and models. Differences between the translated and community-based data highlight the importance of cultural adaptation for multilingual bias evaluation. Fine-tuning reduces bias on average, but not consistently across models and identities. Warning: This text contains examples of anti-queer hateful language and stereotypes.
Abstract:While gender and racial biases in language models have been widely studied, anti-LGBTQ biases remain underexplored, particularly beyond English. Existing benchmarks often do not capture cultural and linguistic variation and rely on gender representations. This paper introduces a multilingual German-English benchmark dataset for the evaluation of anti-LGBTQ biases in language models. It combines community-sourced stereotypes from German-speaking queer individuals with a German translation of WinoQueer. The data is used to evaluate eight language models across sizes and architectures and explore mitigation through fine-tuning on community and progressive media content. Results show that language models reproduce anti-queer stereotypes, with variation across identities and models. Differences between the translated and community-based data highlight the importance of cultural adaptation for multilingual bias evaluation. Fine-tuning reduces bias on average, but not consistently across models and identities. Warning: This text contains examples of anti-queer hateful language and stereotypes.
Computation and Language (cs.CL); Artificial Intelligence (cs.AI)
arXiv:2608.30884 [cs.CL]
(or arXiv:2608.30884v1 [cs.CL] for this version)
https://doi.org/10.48550/arXiv.2608.30884 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 241 cuvinte
Subiecte conexe
WinoQueer-NL: Assessing Bias in Dutch Language Models toward LGBTQ+ Identities
arXiv · 3 sept. 2026
Inference-Time Steering for Cross-Lingual Factual Consistency in LLMs
arXiv · 23 iul. 2026
LLM Analysis of 150+ years of German Parliamentary Debates on Migration Reveals Shift from Post-War Solidarity to Anti-Solidarity in the Last Decade
arXiv · 28 aug. 2026
Language Bias under Conflicting Information in Multilingual LLMs
arXiv · 3 iun. 2026