analiza aiqbench
ReguSim introduces a rigorous evaluation framework for LLM agents in financial markets, distinguishing between stated reasoning and actual execution enforcement to address critical safety gaps in automated trading. By benchmarking models like DeepSeek V4 Pro and Gemini 3.5 Flash against structured baselines, the study demonstrates that simple monitoring systems can often match or exceed complex prompt-only approaches in detecting rule violations. This work shifts the focus of compliance assessment from a single score to an audit of evidence use and action grounding, offering essential insights for deploying reliable AI agents in high-stakes regulatory environments.
ReguSim: Evaluating LLM Agent Rule Grounding in Financial Compliance
arXiv:2608.19974v2 Announce Type: replace Abstract: LLM agents in financial markets may cite rules yet still submit orders that violate executable constraints or misread surveillance evidence. We introduce ReguSim, a controlled financial-compliance environment, and ReguBench, a target-marked monitoring benchmark, to separate four artifacts: stated reasoning, attempted action, execution enforcement, and monitor evidence. In trader runs with DeepSeek V4 Pro and Gemini 3.5 Flash, visible rules reduce but do not eliminate rejected actions, and incentive or persona framing shifts behavior. A bridge study shows that trader rationales can mislead an independent monitor unless enforcement evidence is shown. In monitoring, simple structured baselines either match or exceed prompt-only LLMs. The results frame financial compliance evaluation as an audit of rule-grounded actions and evidence use, rather than a single compliance score.
Abstract:LLM agents in financial markets may cite rules yet still submit orders that violate executable constraints or misread surveillance evidence. We introduce ReguSim, a controlled financial-compliance environment, and ReguBench, a target-marked monitoring benchmark, to separate four artifacts: stated reasoning, attempted action, execution enforcement, and monitor evidence. In trader runs with DeepSeek V4 Pro and Gemini 3.5 Flash, visible rules reduce but do not eliminate rejected actions, and incentive or persona framing shifts behavior. A bridge study shows that trader rationales can mislead an independent monitor unless enforcement evidence is shown. In monitoring, simple structured baselines either match or exceed prompt-only LLMs. The results frame financial compliance evaluation as an audit of rule-grounded actions and evidence use, rather than a single compliance score.
Artificial Intelligence (cs.AI)
arXiv:2608.19974 [cs.AI]
(or arXiv:2608.19974v2 [cs.AI] for this version)
https://doi.org/10.48550/arXiv.2608.19974 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 214 cuvinte
Subiecte conexe

LiveLaw Supreme Court Half-Yearly Digest: January - June 2026 [Part 2]
livelaw.in · 13 sept. 2026

Built Technologies builds an AI-powered document intelligence solution on AWS to power agents across real estate finance
AWS · 15 iul. 2026

Introducing Mistral OCR 4
Mistral AI · 23 iun. 2026

What are AI Hallucinations?
Databricks · 13 aug. 2026