analiza aiqbench
This research identifies a critical nuance in diffusion language models (DLMs) where social bias metrics fluctuate significantly based on specific sampling configurations like block length and remasking strategies. The study demonstrates that fairness assessments for DLMs are inherently dependent on these operational settings, even after applying standard debiasing techniques, challenging the stability of current evaluation frameworks. Consequently, developers must account for sampling-dependent variance when characterizing model safety and bias before deployment.
Beyond Model Weights: Sampling-Dependent Fairness in Diffusion Language Models
arXiv:2610.27584v2 Announce Type: replace Abstract: Diffusion language models (DLMs) are emerging as a practical alternative to autoregressive language models (ARLMs), yet their fairness remains largely uncharacterized. DLMs introduce sampling controls with no direct autoregressive counterpart, such as block length and remasking strategy, that directly set their speed--quality operating point. This raises a basic question: does a DLM's measured social bias remain stable across these settings? Across LLaDA, LLaDA-1.5, and Dream on Open-BBQ, we find that all three models exhibit social bias, but both the magnitude of measured bias and the relative ordering of models by bias are highly sensitive to the sampling configuration. This sampling dependence also changes what successful debiasing means. While mitigation strategies established for ARLMs transfer effectively to DLMs and reduce bias without degrading downstream performance, sampling dependence persists even after mitigation. Our results show that fairness in DLMs cannot be characterized independently of the sampling configuration, even after debiasing. Project page is available at https://pakapaka333.github.io/Beyond-Model-Weights/.
Abstract:Diffusion language models (DLMs) are emerging as a practical alternative to autoregressive language models (ARLMs), yet their fairness remains largely uncharacterized. DLMs introduce sampling controls with no direct autoregressive counterpart, such as block length and remasking strategy, that directly set their speed--quality operating point. This raises a basic question: does a DLM's measured social bias remain stable across these settings? Across LLaDA, LLaDA-1.5, and Dream on Open-BBQ, we find that all three models exhibit social bias, but both the magnitude of measured bias and the relative ordering of models by bias are highly sensitive to the sampling configuration. This sampling dependence also changes what successful debiasing means. While mitigation strategies established for ARLMs transfer effectively to DLMs and reduce bias without degrading downstream performance, sampling dependence persists even after mitigation. Our results show that fairness in DLMs cannot be characterized independently of the sampling configuration, even after debiasing. Project page is available at this https URL.
31 pages, 11 figures, 17 tables. Project page: this https URL ; v2: corrected an author's name in metadata (no change to content)
Computation and Language (cs.CL)
arXiv:2610.27584 [cs.CL]
(or arXiv:2610.27584v2 [cs.CL] for this version)
https://doi.org/10.48550/arXiv.2610.27584 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 272 cuvinte