aiqbench analysis
This research introduces a novel benchmark to evaluate next-action prediction systems specifically for spreadsheets, addressing the current lack of auto-completion features in these ubiquitous tools. By curating synthetic datasets using LLM refinement and proposing an interactive evaluation framework, it establishes new baselines involving zero-shot models and fine-tuned small language models tailored for complex spatial-temporal actions. The work advances practical machine learning applications by bridging the gap between developer-focused code completion and productivity-enhancing spreadsheet automation.
A Benchmark and Framework for Evaluating Next Action Predictions in Spreadsheets
arXiv:2606.13802v1 Announce Type: cross Abstract: Predictive code completion greatly accelerates how quickly developers work. In spreadsheets, despite being much more common, such auto-completion features are virtually non-existent. To address this gap, we introduce a benchmark for systems that observe a sequence of user actions in a spreadsheet and predict future actions. Two challenges are (1) the absence of edit histories in public spreadsheet corpora and (2) the complex space of spreadsheet actions (spatial, temporal, composite). To address (1), we manually curate 52 sequences of 12K actions that recreate spreadsheets from public corpora, seeded by parametrized heuristics and LLM refinement. To address (2), we propose an online evaluation that expects a prediction after each user action, accepts or rejects that prediction, updates the future actions upon acceptance, and repeats this until the target spreadsheet is obtained. We use multiple baseline predictors (including zero-shot LLMs, fine-tuned SLMs, and classical models) and analyze different properties that our benchmark teaches us, including but not limited to: properties of saved actions and false positives, efficiency, effect of user profiles, effect of triggers, and effect of context.
Abstract:Predictive code completion greatly accelerates how quickly developers work. In spreadsheets, despite being much more common, such auto-completion features are virtually non-existent. To address this gap, we introduce a benchmark for systems that observe a sequence of user actions in a spreadsheet and predict future actions. Two challenges are (1) the absence of edit histories in public spreadsheet corpora and (2) the complex space of spreadsheet actions (spatial, temporal, composite). To address (1), we manually curate 52 sequences of 12K actions that recreate spreadsheets from public corpora, seeded by parametrized heuristics and LLM refinement. To address (2), we propose an online evaluation that expects a prediction after each user action, accepts or rejects that prediction, updates the future actions upon acceptance, and repeats this until the target spreadsheet is obtained. We use multiple baseline predictors (including zero-shot LLMs, fine-tuned SLMs, and classical models) and analyze different properties that our benchmark teaches us, including but not limited to: properties of saved actions and false positives, efficiency, effect of user profiles, effect of triggers, and effect of context.
Accepted at ICML 2026. Code and benchmark: this https URL
Software Engineering (cs.SE); Artificial Intelligence (cs.AI); Human-Computer Interaction (cs.HC); Machine Learning (cs.LG)
arXiv:2606.13802 [cs.SE]
(or arXiv:2606.13802v1 [cs.SE] for this version)
https://doi.org/10.48550/arXiv.2606.13802 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 289 words
Related coverage
Tabular LLMs: An Introduction to the Foundation Models That Predict Your Spreadsheet
Towards Data Science · Jul 24, 2026
Structured Prediction for Scalable Spreadsheet Table Understanding: From Cell Types to Table Ranges (Extended Version)
arXiv · Aug 19, 2026
A simple framework for deciding what to automate vs. keep manual
Proton Blog · Oct 12, 2026
Announcing Workforce Management: Forecast, schedule, and plan teammate cover in the Intercom helpdesk
Intercom Blog · Nov 2, 2026