aiqbench analysis
This arXiv preprint proposes an LLM agent harness that stores workflow records in a schema-defined memory and rebuilds a compact context at finite-state-machine transitions, so context stays bounded over long finite element analysis runs. The reported results are 16 of 18 successful runs across six benchmarks, but these are small-scale, self-reported evaluations on one domain, and the paper doesn't yet show how the approach performs against stronger baselines or other simulation tools. For AI readers, the useful idea is treating context management as a designed part of the workflow rather than an afterthought.
MarkovSim: an LLM Agent Harness with Bounded Context for Long-Horizon Computational Simulation
arXiv:2610.35349v1 Announce Type: cross Abstract: Recent studies have applied large language model (LLM) agent systems to computational simulation workflows. Such workflows are long-horizon, so the information generated can exceed what an LLM can reason over at once, its context window. Context management is therefore essential, yet existing systems have either neglected it or treated it as a supporting mechanism rather than a subject of study. This paper presents MarkovSim, an LLM agent harness for long-horizon computational simulation, whose core is a context management scheme built into the workflow. The workflow record is written to a memory system with a predefined schema, and at workflow-defined boundaries, namely the state transitions of a finite-state machine a compact representation of the solution progress is constructed from it as the agent's context. Instantiated on finite element analysis and evaluated on six benchmark problems, MarkovSim passed 16 of 18 runs and corrected its own faults in both the solver operations and the model physics. The engineer was consulted only where engineering judgement was needed. The context remained bounded throughout every run, regardless of its length.
Abstract:Recent studies have applied large language model (LLM) agent systems to computational simulation workflows. Such workflows are long-horizon, so the information generated can exceed what an LLM can reason over at once, its context window. Context management is therefore essential, yet existing systems have either neglected it or treated it as a supporting mechanism rather than a subject of study. This paper presents MarkovSim, an LLM agent harness for long-horizon computational simulation, whose core is a context management scheme built into the workflow. The workflow record is written to a memory system with a predefined schema, and at workflow-defined boundaries, namely the state transitions of a finite-state machine a compact representation of the solution progress is constructed from it as the agent's context. Instantiated on finite element analysis and evaluated on six benchmark problems, MarkovSim passed 16 of 18 runs and corrected its own faults in both the solver operations and the model physics. The engineer was consulted only where engineering judgement was needed. The context remained bounded throughout every run, regardless of its length.
Computational Engineering, Finance, and Science (cs.CE); Artificial Intelligence (cs.AI)
arXiv:2610.35349 [cs.CE]
(or arXiv:2610.35349v1 [cs.CE] for this version)
https://doi.org/10.48550/arXiv.2610.35349 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 275 words