Overlay\_dx - Automating forecasting evaluation

Abstract:Traditional evaluation metrics provides numerical values but often lack comprehensibility, hindering effective differentiation of model performances. Our work addresses this challenge by introducing overlay\_dx, a novel evaluation metric measuring the performance of time series prediction models. Overlay\_dx is a visual metric that represents the percentage of predictions falling within a confidence interval around actual values. Additionally, once evaluation results are plotted, overlay\_dx computes the area under the overlay curve, providing a quantitative measure of alignment between predicted and actual values across different thresholds and predictions. Through extensive experiments, we demonstrate that our approach offers a unified evaluation framework that combines both visual and numerical assessments, enabling improved model comparison and providing valuable insights for further research and optimization efforts in time series prediction.

Machine Learning (cs.LG); Artificial Intelligence (cs.AI)

arXiv:2609.24586 [cs.LG]

(or arXiv:2609.24586v1 [cs.LG] for this version)

https://doi.org/10.48550/arXiv.2609.24586 Focus to learn more arXiv-issued DOI via DataCite (pending registration)

Optimization and Learning. OLA 2025. Communications in Computer and Information Science, Apr 2025, Dubai, United Arab Emirates, France. pp.98-110

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.