aiqbench analysis
This paper uses LLM agents to turn natural-language requests into inspectable, executable computational graphs for pathology image analysis, with the graph as a check against hallucinated operations. The benchmark covers 14 LLMs on 90 unit tasks, and the reported 0.967 success rate applies only to those narrow tasks, not to full end-to-end clinical workflows. For AI readers, the useful takeaway is the separation of choosing the right operation from configuring it correctly, which is where the failures concentrate.
TIAgent: From Natural Language to Grounded Agentic Workflows for Computational Pathology
arXiv:2610.32993v1 Announce Type: cross Abstract: Computational pathology (CPath) analyses increasingly depend on multi-step computational workflows that require substantial programming expertise, limiting their accessibility to pathologists and other domain experts without extensive coding experience. Although large language models (LLMs) provide a promising natural-language interface for constructing such analyses, hallucinated operations and incorrect workflow specifications limit their reliability in scientific settings. We introduce Tissue Image Analysis Agent (TIAgent), an automated CPath workflow generation framework that, given a natural-language prompt specifying a complex analytical task together with an input dataset, automatically constructs a verifiable computational graph employing existing CPath operations. The resulting graph can be inspected and executed, enabling natural-language construction of complex CPath workflows for applications including automated biomarker discovery and tissue analysis. We demonstrate end-to-end workflow generation and execution and systematically benchmark 14 backbone LLMs across 90 unit tasks spanning 18 CPath operation types. The ability of each LLM to generate a complete and correct unit workflow is quantified using Unit Workflow Success (UWS). Performance varied substantially across backbone models, with Kimi-K2.7-Code achieving the highest UWS of 0.967 (95% CI: 0.907-0.989). Analysis of node and parameter correctness further revealed that selecting the appropriate computational operation and correctly configuring it represent distinct challenges for LLM-based workflow generation. TIAgent provides a foundation for making complex CPath workflow construction more accessible while enabling systematic evaluation of LLM-driven scientific workflow generation.
Abstract:Computational pathology (CPath) analyses increasingly depend on multi-step computational workflows that require substantial programming expertise, limiting their accessibility to pathologists and other domain experts without extensive coding experience. Although large language models (LLMs) provide a promising natural-language interface for constructing such analyses, hallucinated operations and incorrect workflow specifications limit their reliability in scientific settings. We introduce Tissue Image Analysis Agent (TIAgent), an automated CPath workflow generation framework that, given a natural-language prompt specifying a complex analytical task together with an input dataset, automatically constructs a verifiable computational graph employing existing CPath operations. The resulting graph can be inspected and executed, enabling natural-language construction of complex CPath workflows for applications including automated biomarker discovery and tissue analysis. We demonstrate end-to-end workflow generation and execution and systematically benchmark 14 backbone LLMs across 90 unit tasks spanning 18 CPath operation types. The ability of each LLM to generate a complete and correct unit workflow is quantified using Unit Workflow Success (UWS). Performance varied substantially across backbone models, with Kimi-K2.7-Code achieving the highest UWS of 0.967 (95% CI: 0.907-0.989). Analysis of node and parameter correctness further revealed that selecting the appropriate computational operation and correctly configuring it represent distinct challenges for LLM-based workflow generation. TIAgent provides a foundation for making complex CPath workflow construction more accessible while enabling systematic evaluation of LLM-driven scientific workflow generation.
Quantitative Methods (q-bio.QM); Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Multiagent Systems (cs.MA)
arXiv:2610.32993 [q-bio.QM]
(or arXiv:2610.32993v1 [q-bio.QM] for this version)
https://doi.org/10.48550/arXiv.2610.32993 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 323 words
Related coverage
Performance Model for Hybrid Quantum-Classical Workflows
arXiv · Jul 20, 2026
Designing Agentic AI Workflow Portfolios under Imperfect Selection and Compute Cost
arXiv · Sep 17, 2026
OceanMind: A multi-agent AI system for ocean diagnosis
arXiv · Oct 7, 2026
Autonomous Chemical Mechanistic Discovery through Agentic Reasoning and Validation
arXiv · Sep 12, 2026