aiqbench analysis
This paper addresses critical fragility in temporal cascade prediction by introducing a new benchmarking protocol that eliminates data leakage and accounts for performance inflation. By releasing the Taoke dataset with real-world conversion labels, it shifts the field from leaderboard gaming toward reliable, fidelity-aware evaluation of forecasting models. The work establishes essential standards for rigorously assessing complex machine learning systems in dynamic network environments.
From Leakage to Fidelity: Reliable Benchmarking for Temporal Cascade Prediction
arXiv:2510.25348v3 Announce Type: replace Abstract: Temporal cascade prediction is widely studied, yet its empirical foundations remain fragile. Most existing works report results under random cascade splits that mix past and future signals, rely on datasets with limited features and no downstream conversion labels, and compare increasingly complex models without systematically examining whether benchmark conclusions are protocol-dependent. This paper argues that the field should move from leakage-prone evaluation toward fidelity-aware benchmarking. We introduce a protocol suite and renewed evaluation standard for temporal cascade prediction, centered on the Full Temporal protocol, overlap-based leakage diagnostics, and analyses of performance inflation and temporal drift. To broaden the scope of benchmark tasks, we also present Taoke, a real-world e-commerce cascade dataset with rich promoter/product features and observed purchase conversions, enabling both first-stage popularity forecasting and second-stage conversion forecasting under a shared benchmark asset. Finally, we include CasTemp as a lightweight reference method and additionally probe a larger same-task internal extension to verify that this pipeline remains operational at substantially greater scale. Together, these components turn cascade prediction from a protocol-sensitive leaderboard exercise into a more reliable analysis and benchmarking problem, while still providing a practical reference pipeline for large-scale evaluation and conversion-aware modeling.
Abstract:Temporal cascade prediction is widely studied, yet its empirical foundations remain fragile. Most existing works report results under random cascade splits that mix past and future signals, rely on datasets with limited features and no downstream conversion labels, and compare increasingly complex models without systematically examining whether benchmark conclusions are protocol-dependent. This paper argues that the field should move from leakage-prone evaluation toward fidelity-aware benchmarking. We introduce a protocol suite and renewed evaluation standard for temporal cascade prediction, centered on the Full Temporal protocol, overlap-based leakage diagnostics, and analyses of performance inflation and temporal drift. To broaden the scope of benchmark tasks, we also present Taoke, a real-world e-commerce cascade dataset with rich promoter/product features and observed purchase conversions, enabling both first-stage popularity forecasting and second-stage conversion forecasting under a shared benchmark asset. Finally, we include CasTemp as a lightweight reference method and additionally probe a larger same-task internal extension to verify that this pipeline remains operational at substantially greater scale. Together, these components turn cascade prediction from a protocol-sensitive leaderboard exercise into a more reliable analysis and benchmarking problem, while still providing a practical reference pipeline for large-scale evaluation and conversion-aware modeling.
Machine Learning (cs.LG); Social and Information Networks (cs.SI)
arXiv:2510.25348 [cs.LG]
(or arXiv:2510.25348v3 [cs.LG] for this version)
https://doi.org/10.48550/arXiv.2510.25348 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 290 words
Related coverage
Beneath the Scores: Rethinking Hallucination Evaluation for Video Understanding Models
arXiv · Sep 25, 2026
Support-Set Target Leakage in Relational Foundation Models during In-Context Learning: Model Dependence and Evaluation Reliability
arXiv · Sep 30, 2026
Runway’s WorldPrompt and the Engineering of Real-Time Worlds
Latent Space · Sep 25, 2026
BWM: A Low-Cost High-Fidelity World Simulator for Robot Learning
arXiv · Aug 3, 2026