What Does Disagreement Mean? A Semantic Identifiability Frontier for Pairwise Preference Learning

Title:What Does Disagreement Mean? A Semantic Identifiability Frontier for Pairwise Preference Learning

Abstract:Pairwise comparison data are a standard supervision signal in preference learning, but disagreement in such data can reflect stable latent heterogeneity, ephemeral stochasticity, or non-binary responses hidden by a forced-choice interface. We study which of these semantic distinctions are identifiable from the feedback channel itself. We introduce a nonparametric latent semantic model with three querywise quantities: aggregate directional preference $\mu_q$, persistent disagreement $\tau_q$, and non-binary mass $\alpha_q$. Under the anonymous binary channel, we prove an exact semantic fiber theorem: $\mu_q$ is point-identified, whereas $(\tau_q, \alpha_q)$ vary over an explicit feasible region compatible with the same binary law. Same-annotator repeats identify $\tau_q$ through a covariance identity, abstention-enabled feedback identifies $\alpha_q$ directly, and together they point-identify the full semantic triple. We also derive finite-sample estimators, nonasymptotic concentration bounds, and semantics-aware action-recovery guarantees, and validate the theory on theorem-matched synthetic data.

53 pages. Best Paper Award at the 2nd Workshop on Epistemic Intelligence in Machine Learning (EIML), ICML 2026

Computer Science and Game Theory (cs.GT); Machine Learning (cs.LG)

arXiv:2610.20450 [cs.GT]

(or arXiv:2610.20450v1 [cs.GT] for this version)

https://doi.org/10.48550/arXiv.2610.20450 Focus to learn more arXiv-issued DOI via DataCite (pending registration)

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.