analiza aiqbench
This paper shows that a demographic parity constraint enforced on predicted outcomes can look satisfied at decision time while leaving a large share of the true-outcome disparity intact. In the US Census income experiments, roughly a third of the original gap remained, and the remainder was invisible from the predictions alone. For ML practitioners, the practical takeaway is that fairness audits of predict-then-optimize systems need access to ground-truth outcomes or explicit error modeling, not just metrics computed on predictions.
Fairness under Prediction Error in Predict-then-Optimize Pipelines
arXiv:2610.18593v1 Announce Type: new Abstract: Fairness constraints in predict-then-optimize pipelines are necessarily enforced on predicted outcomes $\hat{y}$, since true outcomes $y$ are unobserved at decision time. We study whether a demographic parity constraint enforced this way continues to hold against $y$, and how the resulting violation scales with prediction error. Using a budget-constrained allocation problem solved as a linear program, we show empirically that the true-outcome fairness gap grows with prediction error $\sigma$ and saturates, well fit by $c(1 - e^{-\sigma/\tau})$ with $c = 0.1545 \pm 0.0103$ and $\tau = 3.9045 \pm 0.3745$; its mean is invariant to sample size while its variance decays at the standard $O(1/\sqrt{n})$ rate. We derive a two-part theoretical bound: a closed-form local sensitivity result showing the gap's first-order response to prediction error depends on only two individuals per group, validated against the solver to within three significant figures; and a universal, error-independent ceiling following from the boundedness of the decision variable, confirmed consistent with but looser than the empirical saturation level. On US Census income data, enforcing demographic parity on a trained predictor's output reduces an unconstrained disparity in true income to roughly one-third of its original size, leaving 34.4% $\pm$ 3.3% (California) and 35.7% $\pm$ 1.8% (Texas) of the disparity intact and undetectable from the predictions alone -- consistent not only in mechanism but in approximate magnitude across two states with different demographics and income distributions, and stable across ten independent train/test splits in each. A fairness constraint that appears fully satisfied at decision time can mask a substantial, systematic violation of the outcome it was meant to protect.
Abstract:Fairness constraints in predict-then-optimize pipelines are necessarily enforced on predicted outcomes $\hat{y}$, since true outcomes $y$ are unobserved at decision time. We study whether a demographic parity constraint enforced this way continues to hold against $y$, and how the resulting violation scales with prediction error. Using a budget-constrained allocation problem solved as a linear program, we show empirically that the true-outcome fairness gap grows with prediction error $\sigma$ and saturates, well fit by $c(1 - e^{-\sigma/\tau})$ with $c = 0.1545 \pm 0.0103$ and $\tau = 3.9045 \pm 0.3745$; its mean is invariant to sample size while its variance decays at the standard $O(1/\sqrt{n})$ rate. We derive a two-part theoretical bound: a closed-form local sensitivity result showing the gap's first-order response to prediction error depends on only two individuals per group, validated against the solver to within three significant figures; and a universal, error-independent ceiling following from the boundedness of the decision variable, confirmed consistent with but looser than the empirical saturation level. On US Census income data, enforcing demographic parity on a trained predictor's output reduces an unconstrained disparity in true income to roughly one-third of its original size, leaving 34.4% $\pm$ 3.3% (California) and 35.7% $\pm$ 1.8% (Texas) of the disparity intact and undetectable from the predictions alone -- consistent not only in mechanism but in approximate magnitude across two states with different demographics and income distributions, and stable across ten independent train/test splits in each. A fairness constraint that appears fully satisfied at decision time can mask a substantial, systematic violation of the outcome it was meant to protect.
Machine Learning (cs.LG); Computers and Society (cs.CY)
arXiv:2610.18593 [cs.LG]
(or arXiv:2610.18593v1 [cs.LG] for this version)
https://doi.org/10.48550/arXiv.2610.18593 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 359 cuvinte
Subiecte conexe
Handling Imbalanced Classification: What Works Better Than SMOTE
Analytics Vidhya · 12 iul. 2026
Runway’s WorldPrompt and the Engineering of Real-Time Worlds
Latent Space · 25 sept. 2026
[AINews] not much happened today
Latent Space · 18 sept. 2026
Building Blocks for Foundation Model Training and Inference on AWS
Hugging Face · 11 mai 2026