Personalized Execution Time Optimization for Billion-Scale Scheduled Jobs

Abstract:Scheduled batch jobs are widely used on asynchronous computing platforms to execute enterprise applications such as promotional notifications and candidate pre-computation for recommender systems. Delivering or updating information at the right time is important for user experience and execution impact, yet providing a versatile, personalized execution time optimization solution across diverse product scenarios while maintaining reasonable infrastructure costs remains challenging. In this paper, we present a deployed system that serves billions of users daily, combining learning-to-rank with a "best time policy" for execution time selection. We describe the four-stage evolution of our approach: from heuristic peak-hour rules, to pointwise ML-based activity pattern predictions, to a linear signal assembler with globally fixed weights, and finally to a contextual ensemble learner that produces per-user adaptive fusion weights via a neural policy network trained with listwise learning-to-rank objectives. We further report the discovery of cross-use-case cannibalization effects and introduce a coordination system to mitigate the problem. Our production experiments demonstrate measurable improvements in both execution efficiency and downstream product impact. We share deployment lessons including failure analyses and design decisions accumulated over four years of operating this system at scale. To our knowledge, this represents the first ML-based multi-tenant execution time optimization system deployed across different product domains at industrial scale.

Machine Learning (cs.LG); Distributed, Parallel, and Cluster Computing (cs.DC)

arXiv:2203.06158 [cs.LG]

(or arXiv:2203.06158v3 [cs.LG] for this version)

https://doi.org/10.48550/arXiv.2203.06158 Focus to learn more arXiv-issued DOI via DataCite

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.