From Leakage to Fidelity: Reliable Benchmarking for Temporal Cascade Prediction

arXiv:2510.25348v3 Announce Type: replace Abstract: Temporal cascade prediction is widely studied, yet its empirical foundations remain fragile. Most existing works report results under random cascade splits that mix past and future signals, rely on datasets with limited features and no downstream conversion labels, and compare increasingly complex models without systematically examining whether benchmark conclusions are protocol-dependent. This paper argues that the field should move from leakage-prone evaluation toward fidelity-aware benchmarking. We introduce a protocol suite and renewed evaluation standard for temporal cascade prediction, centered on the Full Temporal protocol, overlap-based leakage diagnostics, and analyses of performance inflation and temporal drift. To broaden the scope of benchmark tasks, we also present Taoke, a real-world e-commerce cascade dataset with rich promoter/product features and observed purchase conversions, enabling both first-stage popularity forecasting and second-stage conversion forecasting under a shared benchmark asset. Finally, we include CasTemp as a lightweight reference method and additionally probe a larger same-task internal extension to verify that this pipeline remains operational at substantially greater scale. Together, these components turn cascade prediction from a protocol-sensitive leaderboard exercise into a more reliable analysis and benchmarking problem, while still providing a practical reference pipeline for large-scale evaluation and conversion-aware modeling.

Abstract:Temporal cascade prediction is widely studied, yet its empirical foundations remain fragile. Most existing works report results under random cascade splits that mix past and future signals, rely on datasets with limited features and no downstream conversion labels, and compare increasingly complex models without systematically examining whether benchmark conclusions are protocol-dependent. This paper argues that the field should move from leakage-prone evaluation toward fidelity-aware benchmarking. We introduce a protocol suite and renewed evaluation standard for temporal cascade prediction, centered on the Full Temporal protocol, overlap-based leakage diagnostics, and analyses of performance inflation and temporal drift. To broaden the scope of benchmark tasks, we also present Taoke, a real-world e-commerce cascade dataset with rich promoter/product features and observed purchase conversions, enabling both first-stage popularity forecasting and second-stage conversion forecasting under a shared benchmark asset. Finally, we include CasTemp as a lightweight reference method and additionally probe a larger same-task internal extension to verify that this pipeline remains operational at substantially greater scale. Together, these components turn cascade prediction from a protocol-sensitive leaderboard exercise into a more reliable analysis and benchmarking problem, while still providing a practical reference pipeline for large-scale evaluation and conversion-aware modeling.

Machine Learning (cs.LG); Social and Information Networks (cs.SI)

arXiv:2510.25348 [cs.LG]

(or arXiv:2510.25348v3 [cs.LG] for this version)

https://doi.org/10.48550/arXiv.2510.25348 Focus to learn more arXiv-issued DOI via DataCite

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.

analiza aiqbench

From Leakage to Fidelity: Reliable Benchmarking for Temporal Cascade Prediction

Subiecte conexe