Fairness Is Not Enough: Auditing Competence and Intersectional Bias in AI-powered Resume Screening

arXiv:2507.11548 (cs)

Abstract:The use of publicly available generative AI systems for resume evaluation is often justified by the assumption that these tools reduce bias relative to human judgment. However, this framing leaves a prior question unresolved: whether these systems are capable of performing the evaluative task at all. This study presents a two-part audit of eight widely used AI platforms used for resume screening. Drawing on the concept of the Illusion of Neutrality, the study examines cases in which systems appear demographically unbiased because they lack the ability to meaningfully differentiate among candidates. Experiment 1 evaluates racial and gender bias using matched fictitious resumes and finds that bias persists in context-dependent and intersectional forms. Some models penalize candidates for the presence of demographic signals, while others exhibit inconsistent patterns across roles and identities under controlled conditions. Experiment 2 evaluates task competence using resume-role mismatch and keyword-manipulation tests and finds that several models that appear relatively unbiased fail to distinguish relevant from irrelevant candidate experience in relation to the target role. In these cases, outputs appear stable or neutral not because the systems are fair, but because they fail to perform meaningful evaluation. Fairness assessments alone are therefore insufficient. The paper proposes a dual-validation framework that requires auditing for both demographic bias and evaluative competence as a minimum condition for responsible deployment.

Computers and Society (cs.CY); Artificial Intelligence (cs.AI); Computation and Language (cs.CL)

I.2.1; K.4.2; I.2.6; K.4.1

arXiv:2507.11548 [cs.CY]

(or arXiv:2507.11548v3 [cs.CY] for this version)

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.