Loss Knows Best: Detecting Annotation Errors in Videos via Loss Trajectories

arXiv:2602.15154v2 Announce Type: replace-cross Abstract: Reliable video understanding requires high-quality video datasets that can provide both precise semantic labels and temporally consistent annotations. Detecting annotation errors in densely labeled videos is challenging because errors may arise from semantic **mislabeling**, where labels disagree with visual content, or temporal **disordering**, where otherwise plausible labels violate procedural progression. Training dynamics have been used to identify mislabeled training examples primarily for static samples. We investigate checkpoint loss dynamics for **out-of-sample auditing** of temporally annotated videos. We compute **Cumulative Sample Loss (CSL)** as the mean annotation-conditioned loss of an audit frame across checkpoints trained on a *disjoint* reference set. CSL acts as a dynamic fingerprint and captures the persistent disagreement between its annotation and learned visual-temporal structure. High-CSL frames are then flagged as likely candidates for potential annotation errors, including semantic mislabeling or temporal disordering. Experiments on EgoPER and Cholec80 show that CSL substantially outperforms final-checkpoint loss and achieves up to a **4.2-point AUC improvement** over prior baselines on EgoPER and **92.0/78.5 AUC** for mislabeling/disordering on Cholec80. These results demonstrate checkpoint loss dynamics as an effective diagnostic for temporal annotation auditing.

Abstract:Reliable video understanding requires high-quality video datasets that can provide both precise semantic labels and temporally consistent annotations. Detecting annotation errors in densely labeled videos is challenging because errors may arise from semantic **mislabeling**, where labels disagree with visual content, or temporal **disordering**, where otherwise plausible labels violate procedural progression. Training dynamics have been used to identify mislabeled training examples primarily for static samples. We investigate checkpoint loss dynamics for **out-of-sample auditing** of temporally annotated videos. We compute **Cumulative Sample Loss (CSL)** as the mean annotation-conditioned loss of an audit frame across checkpoints trained on a *disjoint* reference set. CSL acts as a dynamic fingerprint and captures the persistent disagreement between its annotation and learned visual-temporal structure. High-CSL frames are then flagged as likely candidates for potential annotation errors, including semantic mislabeling or temporal disordering. Experiments on EgoPER and Cholec80 show that CSL substantially outperforms final-checkpoint loss and achieves up to a **4.2-point AUC improvement** over prior baselines on EgoPER and **92.0/78.5 AUC** for mislabeling/disordering on Cholec80. These results demonstrate checkpoint loss dynamics as an effective diagnostic for temporal annotation auditing.

Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG)

arXiv:2602.15154 [cs.CV]

(or arXiv:2602.15154v2 [cs.CV] for this version)

https://doi.org/10.48550/arXiv.2602.15154 Focus to learn more arXiv-issued DOI via DataCite

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.

aiqbench analysis

Loss Knows Best: Detecting Annotation Errors in Videos via Loss Trajectories

Related coverage