aiqbench analysis
This study challenges the assumption in predictive process monitoring that larger, pre-trained models like LLMs are necessary for next activity prediction tasks. The benchmark reveals that simple counting-based baselines often match or exceed complex architectures across seven real-world event logs, suggesting significant over-engineering in current approaches to sequence modeling within this domain.
David vs. Goliath in Next Activity Prediction: Argmax vs. LSTM, Transformer, and LLM
arXiv:2606.15868v1 Announce Type: new Abstract: Next activity prediction (NAP) is a cornerstone of predictive process monitoring (PPM), enabling organizations to move from retrospective analysis to proactive process steering. The PPM field has progressed from classical machine learning through deep learning architectures such as LSTMs and Transformers to large language models (LLMs). Despite growing model complexity, no benchmark jointly compares LLMs, Transformers, LSTMs, and simple baselines in a direct sequence modeling setting for NAP. In this paper, we fill this gap with a systematic benchmark. We compare vocabulary-adapted LLMs, Transformers trained from scratch, LLM-distilled Transformers, and LSTMs against a simple counting-based argmax baseline across seven real-life event logs. Our results tell a David vs. Goliath story: pretraining confers no consistent improvement over training from scratch, model size shows little effect on performance, and on most datasets the argmax baseline matches or approaches the performance of billion-parameter LLMs.
Title:David vs. Goliath in Next Activity Prediction: Argmax vs. LSTM, Transformer, and LLM
Abstract:Next activity prediction (NAP) is a cornerstone of predictive process monitoring (PPM), enabling organizations to move from retrospective analysis to proactive process steering. The PPM field has progressed from classical machine learning through deep learning architectures such as LSTMs and Transformers to large language models (LLMs). Despite growing model complexity, no benchmark jointly compares LLMs, Transformers, LSTMs, and simple baselines in a direct sequence modeling setting for NAP. In this paper, we fill this gap with a systematic benchmark. We compare vocabulary-adapted LLMs, Transformers trained from scratch, LLM-distilled Transformers, and LSTMs against a simple counting-based argmax baseline across seven real-life event logs. Our results tell a David vs. Goliath story: pretraining confers no consistent improvement over training from scratch, model size shows little effect on performance, and on most datasets the argmax baseline matches or approaches the performance of billion-parameter LLMs.
Machine Learning (cs.LG)
arXiv:2606.15868 [cs.LG]
(or arXiv:2606.15868v1 [cs.LG] for this version)
https://doi.org/10.48550/arXiv.2606.15868 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 248 words
Related coverage
Physics Transformer: Tailoring Transformer for General PDE Prediction
arXiv · Jul 28, 2026
Spatially-Enhanced Temporal Fusion Transformer: Interpretable Multi-Output Prediction for Parametric Dynamical Systems with Time-Varying Inputs
arXiv · Jul 27, 2026
Next-Latent Prediction Transformers Learn Compact World Models
arXiv · Jun 16, 2026
Scaling Vision Transformers Without Training: Prune-and-Merge Token Compression
arXiv · Oct 14, 2026