aiqbench analysis
This research introduces soft label PU learning, a machine learning technique that assigns probabilistic weights to unlabeled samples based on domain knowledge rather than treating them uniformly. By designing novel evaluation metrics specific to this partial labeling scenario, the method effectively optimizes performance in tasks like anti-cheat detection where ground truth data is scarce. The approach represents a significant advancement in handling incomplete datasets within classification problems.
Soft Label PU Learning
arXiv:2405.01990v2 Announce Type: replace Abstract: PU learning refers to the classification problem in which only part of positive samples are labeled. Existing PU learning methods treat unlabeled samples equally. However, in many real tasks, from common sense or domain knowledge, some unlabeled samples are more likely to be positive than others. In this paper, we propose soft label PU learning, in which unlabeled data are assigned soft labels according to their probabilities of being positive. Considering that the ground truth of TPR, FPR, and AUC are unknown, we then design PU counterparts of these metrics to evaluate the performances of soft label PU learning methods within validation data. We show that these new designed PU metrics are good substitutes for the real metrics. After that, a method that optimizes such metrics is proposed. Experiments on public datasets and real datasets for anti-cheat services from Tencent games demonstrate the effectiveness of our proposed method.
Abstract:PU learning refers to the classification problem in which only part of positive samples are labeled. Existing PU learning methods treat unlabeled samples equally. However, in many real tasks, from common sense or domain knowledge, some unlabeled samples are more likely to be positive than others. In this paper, we propose soft label PU learning, in which unlabeled data are assigned soft labels according to their probabilities of being positive. Considering that the ground truth of TPR, FPR, and AUC are unknown, we then design PU counterparts of these metrics to evaluate the performances of soft label PU learning methods within validation data. We show that these new designed PU metrics are good substitutes for the real metrics. After that, a method that optimizes such metrics is proposed. Experiments on public datasets and real datasets for anti-cheat services from Tencent games demonstrate the effectiveness of our proposed method.
Machine Learning (cs.LG)
arXiv:2405.01990 [cs.LG]
(or arXiv:2405.01990v2 [cs.LG] for this version)
https://doi.org/10.48550/arXiv.2405.01990 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 240 words
Related coverage
An Assessment of Human vs. Model Uncertainty in Soft-Label Learning and Calibration
arXiv · Oct 7, 2026
Bayes-Optimal BER and AUC: Estimation and Evaluation of Estimators
arXiv · Sep 3, 2026
BioSentinel at EXIST 2026: Soft-Label Optimization with XLM-RoBERTa for Sexism Intent Classification in Memes
arXiv · Jul 28, 2026
Distributions In, Distributions Out: The Case for Soft-Label Training
arXiv · Jul 31, 2026