aiqbench analysis
This position paper proposes a standardized framework for post-publication code verification in machine learning, aiming to formalize reproducibility efforts beyond current data availability requirements. By suggesting modifications to existing verification badges from ACM and IEEE Access, the authors outline a community-driven system where independent researchers can submit replications to earn visible metadata endorsements. This initiative directly addresses the critical challenge of validating ML research outcomes through structured, peer-reviewed replication processes.
Standardization of Post-Publication Code Verification is Possible with the Support of the Community
arXiv:2601.07189v2 Announce Type: replace Abstract: Reproducibility remains a challenge in machine learning research. While code and data availability requirements have become increasingly common, post-publication verification is still limited and unformalized. This position paper argues that journals and conference proceedings should all implement a post-publication verification submission system. We propose a modification to ACM and IEEE Access verification badges that allows independent researchers to submit post-publication code replications directly to the journal or proceedings, leading to visible verification badges included in the article metadata. Each article may earn up to two badges, each linked to verified code in its corresponding public repository. This framework would be a step forward toward a standardized reproducibility policy and would provide young researchers with an opportunity to improve their curriculum. We describe the motivation, related initiatives, a formal framework, the potential impact, possible limitations, and alternative views.
Abstract:Reproducibility remains a challenge in machine learning research. While code and data availability requirements have become increasingly common, post-publication verification is still limited and unformalized. This position paper argues that journals and conference proceedings should all implement a post-publication verification submission system. We propose a modification to ACM and IEEE Access verification badges that allows independent researchers to submit post-publication code replications directly to the journal or proceedings, leading to visible verification badges included in the article metadata. Each article may earn up to two badges, each linked to verified code in its corresponding public repository. This framework would be a step forward toward a standardized reproducibility policy and would provide young researchers with an opportunity to improve their curriculum. We describe the motivation, related initiatives, a formal framework, the potential impact, possible limitations, and alternative views.
14 pages, 2 figures. Revised and expanded version. To appear at NeurIPS 2026
Machine Learning (cs.LG)
I.2.0; K.6.0; K.7.4; I.5.m
arXiv:2601.07189 [cs.LG]
(or arXiv:2601.07189v2 [cs.LG] for this version)
https://doi.org/10.48550/arXiv.2601.07189 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 246 words
