aiqbench analysis
This research demonstrates that implicit demographic cues embedded in names systematically alter the generative probability distributions of language models, revealing a distinct entropy gap between Black-associated and White-associated identities. Crucially, the study finds that instruction tuning fails to close this racial disparity and highlights how probing methodology fundamentally dictates whether these biases are observed or masked. The findings underscore persistent structural inequities within model architectures that persist regardless of standard alignment techniques.
Token-Level Entropy Reveals Demographic Disparities in Language Models
arXiv:2501.19337v4 Announce Type: replace Abstract: We ask whether demographic identity, signaled by a name alone, systematically reshapes the generative distribution of a language model. Measuring full-vocabulary Shannon entropy at temperature zero across six open-weight base models and 5,760 implicit sentence-completion prompts (e.g., "Tanisha walked into the office on a Monday morning and"), we find that Black-associated names produce higher first-token entropy than White-associated names across all six architectures - opposite to the output-level homogeneity bias documented under explicit demographic prompting (Lee et al., 2024) - and Black-associated names always produce greater entropy above identity-neutral baselines than White-associated names ($\Delta\Delta > 0$ in all six models). Women-associated names co-occur with lower first-token entropy (DL-pooled $\hat\beta = -0.041, p = .019$) and more homogeneous outputs ($\hat\alpha = +0.024, p < .001$) than men-associated names - a pattern convergent with homogeneity bias; race and gender effects are additive. Instruction tuning does not attenuate the race gap (matched-format DL-pooled $\hat{\beta}=+0.153$). Running the same templates with explicit group labels instead of names yields null race effects in 10 of 12 models where implicit probing is significant - establishing that probing methodology is a primary determinant of which distributional structure is recovered.
Abstract:We ask whether demographic identity, signaled by a name alone, systematically reshapes the generative distribution of a language model. Measuring full-vocabulary Shannon entropy at temperature zero across six open-weight base models and 5,760 implicit sentence-completion prompts (e.g., "Tanisha walked into the office on a Monday morning and"), we find that Black-associated names produce higher first-token entropy than White-associated names across all six architectures - opposite to the output-level homogeneity bias documented under explicit demographic prompting (Lee et al., 2024) - and Black-associated names always produce greater entropy above identity-neutral baselines than White-associated names ($\Delta\Delta > 0$ in all six models). Women-associated names co-occur with lower first-token entropy (DL-pooled $\hat\beta = -0.041, p = .019$) and more homogeneous outputs ($\hat\alpha = +0.024, p < .001$) than men-associated names - a pattern convergent with homogeneity bias; race and gender effects are additive. Instruction tuning does not attenuate the race gap (matched-format DL-pooled $\hat{\beta}=+0.153$). Running the same templates with explicit group labels instead of names yields null race effects in 10 of 12 models where implicit probing is significant - establishing that probing methodology is a primary determinant of which distributional structure is recovered.
Computation and Language (cs.CL); Computer Vision and Pattern Recognition (cs.CV)
arXiv:2501.19337 [cs.CL]
(or arXiv:2501.19337v4 [cs.CL] for this version)
From: Messi H.J. Lee [view email] [v1] Fri, 31 Jan 2025 17:36:12 UTC (1,200 KB) [v2] Sat, 24 May 2025 20:54:42 UTC (800 KB) [v3] Thu, 7 May 2026 05:09:44 UTC (861 KB) [v4] Mon, 15 Jun 2026 04:27:33 UTC (853 KB)
Full text under the publisher's open licence · 251 words
Related coverage
Token-Level Entropy Reveals Demographic Disparities in Large Language Models
arXiv · Jul 25, 2026
Do small language models know what they don't know?
arXiv · Sep 21, 2026
Links #3: 2026/06 Part 1
LessWrong · Jun 15, 2026
TRE: Training-Free Hallucination Detection for Diffusion Language Models
arXiv · Jul 28, 2026